diff --git "a/README.md" "b/README.md" deleted file mode 100644--- "a/README.md" +++ /dev/null @@ -1,18484 +0,0 @@ ---- -language: -- multilingual -- af -- am -- ar -- as -- az -- be -- bg -- bn -- br -- bs -- ca -- cs -- cy -- da -- de -- el -- en -- eo -- es -- et -- eu -- fa -- fi -- fr -- fy -- ga -- gd -- gl -- gu -- ha -- he -- hi -- hr -- hu -- hy -- id -- is -- it -- ja -- jv -- ka -- kk -- km -- kn -- ko -- ku -- ky -- la -- lo -- lt -- lv -- mg -- mk -- ml -- mn -- mr -- ms -- my -- ne -- nl -- 'no' -- om -- or -- pa -- pl -- ps -- pt -- ro -- ru -- sa -- sd -- si -- sk -- sl -- so -- sq -- sr -- su -- sv -- sw -- ta -- te -- th -- tl -- tr -- ug -- uk -- ur -- uz -- vi -- xh -- yi -- zh -license: mit -model-index: -- name: intfloat/multilingual-e5-small - results: - - dataset: - config: en - name: MTEB AmazonCounterfactualClassification (en) - revision: e8379541af4e31359cca9fbcf4b00f2671dba205 - split: test - type: mteb/amazon_counterfactual - metrics: - - type: accuracy - value: 73.79104477611939 - - type: ap - value: 36.9996434842022 - - type: f1 - value: 67.95453679103099 - task: - type: Classification - - dataset: - config: de - name: MTEB AmazonCounterfactualClassification (de) - revision: e8379541af4e31359cca9fbcf4b00f2671dba205 - split: test - type: mteb/amazon_counterfactual - metrics: - - type: accuracy - value: 71.64882226980728 - - type: ap - value: 82.11942130026586 - - type: f1 - value: 69.87963421606715 - task: - type: Classification - - dataset: - config: en-ext - name: MTEB AmazonCounterfactualClassification (en-ext) - revision: e8379541af4e31359cca9fbcf4b00f2671dba205 - split: test - type: mteb/amazon_counterfactual - metrics: - - type: accuracy - value: 75.8095952023988 - - type: ap - value: 24.46869495579561 - - type: f1 - value: 63.00108480037597 - task: - type: Classification - - dataset: - config: ja - name: MTEB AmazonCounterfactualClassification (ja) - revision: e8379541af4e31359cca9fbcf4b00f2671dba205 - split: test - type: mteb/amazon_counterfactual - metrics: - - type: accuracy - value: 64.186295503212 - - type: ap - value: 15.496804690197042 - - type: f1 - value: 52.07153895475031 - task: - type: Classification - - dataset: - config: default - name: MTEB AmazonPolarityClassification - revision: e2d317d38cd51312af73b3d32a06d1a08b442046 - split: test - type: mteb/amazon_polarity - metrics: - - type: accuracy - value: 88.699325 - - type: ap - value: 85.27039559917269 - - type: f1 - value: 88.65556295032513 - task: - type: Classification - - dataset: - config: en - name: MTEB AmazonReviewsClassification (en) - revision: 1399c76144fd37290681b995c656ef9b2e06e26d - split: test - type: mteb/amazon_reviews_multi - metrics: - - type: accuracy - value: 44.69799999999999 - - type: f1 - value: 43.73187348654165 - task: - type: Classification - - dataset: - config: de - name: MTEB AmazonReviewsClassification (de) - revision: 1399c76144fd37290681b995c656ef9b2e06e26d - split: test - type: mteb/amazon_reviews_multi - metrics: - - type: accuracy - value: 40.245999999999995 - - type: f1 - value: 39.3863530637684 - task: - type: Classification - - dataset: - config: es - name: MTEB AmazonReviewsClassification (es) - revision: 1399c76144fd37290681b995c656ef9b2e06e26d - split: test - type: mteb/amazon_reviews_multi - metrics: - - type: accuracy - value: 40.394 - - type: f1 - value: 39.301223469483446 - task: - type: Classification - - dataset: - config: fr - name: MTEB AmazonReviewsClassification (fr) - revision: 1399c76144fd37290681b995c656ef9b2e06e26d - split: test - type: mteb/amazon_reviews_multi - metrics: - - type: accuracy - value: 38.864 - - type: f1 - value: 37.97974261868003 - task: - type: Classification - - dataset: - config: ja - name: MTEB AmazonReviewsClassification (ja) - revision: 1399c76144fd37290681b995c656ef9b2e06e26d - split: test - type: mteb/amazon_reviews_multi - metrics: - - type: accuracy - value: 37.682 - - type: f1 - value: 37.07399369768313 - task: - type: Classification - - dataset: - config: zh - name: MTEB AmazonReviewsClassification (zh) - revision: 1399c76144fd37290681b995c656ef9b2e06e26d - split: test - type: mteb/amazon_reviews_multi - metrics: - - type: accuracy - value: 37.504 - - type: f1 - value: 36.62317273874278 - task: - type: Classification - - dataset: - config: default - name: MTEB ArguAna - revision: None - split: test - type: arguana - metrics: - - type: map_at_1 - value: 19.061 - - type: map_at_10 - value: 31.703 - - type: map_at_100 - value: 32.967 - - type: map_at_1000 - value: 33.001000000000005 - - type: map_at_3 - value: 27.466 - - type: map_at_5 - value: 29.564 - - type: mrr_at_1 - value: 19.559 - - type: mrr_at_10 - value: 31.874999999999996 - - type: mrr_at_100 - value: 33.146 - - type: mrr_at_1000 - value: 33.18 - - type: mrr_at_3 - value: 27.667 - - type: mrr_at_5 - value: 29.74 - - type: ndcg_at_1 - value: 19.061 - - type: ndcg_at_10 - value: 39.062999999999995 - - type: ndcg_at_100 - value: 45.184000000000005 - - type: ndcg_at_1000 - value: 46.115 - - type: ndcg_at_3 - value: 30.203000000000003 - - type: ndcg_at_5 - value: 33.953 - - type: precision_at_1 - value: 19.061 - - type: precision_at_10 - value: 6.279999999999999 - - type: precision_at_100 - value: 0.9129999999999999 - - type: precision_at_1000 - value: 0.099 - - type: precision_at_3 - value: 12.706999999999999 - - type: precision_at_5 - value: 9.431000000000001 - - type: recall_at_1 - value: 19.061 - - type: recall_at_10 - value: 62.802 - - type: recall_at_100 - value: 91.323 - - type: recall_at_1000 - value: 98.72 - - type: recall_at_3 - value: 38.122 - - type: recall_at_5 - value: 47.155 - task: - type: Retrieval - - dataset: - config: default - name: MTEB ArxivClusteringP2P - revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d - split: test - type: mteb/arxiv-clustering-p2p - metrics: - - type: v_measure - value: 39.22266660528253 - task: - type: Clustering - - dataset: - config: default - name: MTEB ArxivClusteringS2S - revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 - split: test - type: mteb/arxiv-clustering-s2s - metrics: - - type: v_measure - value: 30.79980849482483 - task: - type: Clustering - - dataset: - config: default - name: MTEB AskUbuntuDupQuestions - revision: 2000358ca161889fa9c082cb41daa8dcfb161a54 - split: test - type: mteb/askubuntudupquestions-reranking - metrics: - - type: map - value: 57.8790068352054 - - type: mrr - value: 71.78791276436706 - task: - type: Reranking - - dataset: - config: default - name: MTEB BIOSSES - revision: d3fb88f8f02e40887cd149695127462bbcf29b4a - split: test - type: mteb/biosses-sts - metrics: - - type: cos_sim_pearson - value: 82.36328364043163 - - type: cos_sim_spearman - value: 82.26211536195868 - - type: euclidean_pearson - value: 80.3183865039173 - - type: euclidean_spearman - value: 79.88495276296132 - - type: manhattan_pearson - value: 80.14484480692127 - - type: manhattan_spearman - value: 80.39279565980743 - task: - type: STS - - dataset: - config: de-en - name: MTEB BUCC (de-en) - revision: d51519689f32196a32af33b075a01d0e7c51e252 - split: test - type: mteb/bucc-bitext-mining - metrics: - - type: accuracy - value: 98.0375782881002 - - type: f1 - value: 97.86012526096033 - - type: precision - value: 97.77139874739039 - - type: recall - value: 98.0375782881002 - task: - type: BitextMining - - dataset: - config: fr-en - name: MTEB BUCC (fr-en) - revision: d51519689f32196a32af33b075a01d0e7c51e252 - split: test - type: mteb/bucc-bitext-mining - metrics: - - type: accuracy - value: 93.35241030156286 - - type: f1 - value: 92.66050333846944 - - type: precision - value: 92.3306919069631 - - type: recall - value: 93.35241030156286 - task: - type: BitextMining - - dataset: - config: ru-en - name: MTEB BUCC (ru-en) - revision: d51519689f32196a32af33b075a01d0e7c51e252 - split: test - type: mteb/bucc-bitext-mining - metrics: - - type: accuracy - value: 94.0699688257707 - - type: f1 - value: 93.50236693222492 - - type: precision - value: 93.22791825424315 - - type: recall - value: 94.0699688257707 - task: - type: BitextMining - - dataset: - config: zh-en - name: MTEB BUCC (zh-en) - revision: d51519689f32196a32af33b075a01d0e7c51e252 - split: test - type: mteb/bucc-bitext-mining - metrics: - - type: accuracy - value: 89.25750394944708 - - type: f1 - value: 88.79234684921889 - - type: precision - value: 88.57293312269616 - - type: recall - value: 89.25750394944708 - task: - type: BitextMining - - dataset: - config: default - name: MTEB Banking77Classification - revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 - split: test - type: mteb/banking77 - metrics: - - type: accuracy - value: 79.41558441558442 - - type: f1 - value: 79.25886487487219 - task: - type: Classification - - dataset: - config: default - name: MTEB BiorxivClusteringP2P - revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40 - split: test - type: mteb/biorxiv-clustering-p2p - metrics: - - type: v_measure - value: 35.747820820329736 - task: - type: Clustering - - dataset: - config: default - name: MTEB BiorxivClusteringS2S - revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908 - split: test - type: mteb/biorxiv-clustering-s2s - metrics: - - type: v_measure - value: 27.045143830596146 - task: - type: Clustering - - dataset: - config: default - name: MTEB CQADupstackRetrieval - revision: None - split: test - type: BeIR/cqadupstack - metrics: - - type: map_at_1 - value: 24.252999999999997 - - type: map_at_10 - value: 31.655916666666666 - - type: map_at_100 - value: 32.680749999999996 - - type: map_at_1000 - value: 32.79483333333334 - - type: map_at_3 - value: 29.43691666666666 - - type: map_at_5 - value: 30.717416666666665 - - type: mrr_at_1 - value: 28.602750000000004 - - type: mrr_at_10 - value: 35.56875 - - type: mrr_at_100 - value: 36.3595 - - type: mrr_at_1000 - value: 36.427749999999996 - - type: mrr_at_3 - value: 33.586166666666664 - - type: mrr_at_5 - value: 34.73641666666666 - - type: ndcg_at_1 - value: 28.602750000000004 - - type: ndcg_at_10 - value: 36.06933333333334 - - type: ndcg_at_100 - value: 40.70141666666667 - - type: ndcg_at_1000 - value: 43.24341666666667 - - type: ndcg_at_3 - value: 32.307916666666664 - - type: ndcg_at_5 - value: 34.129999999999995 - - type: precision_at_1 - value: 28.602750000000004 - - type: precision_at_10 - value: 6.097666666666667 - - type: precision_at_100 - value: 0.9809166666666668 - - type: precision_at_1000 - value: 0.13766666666666663 - - type: precision_at_3 - value: 14.628166666666667 - - type: precision_at_5 - value: 10.266916666666667 - - type: recall_at_1 - value: 24.252999999999997 - - type: recall_at_10 - value: 45.31916666666667 - - type: recall_at_100 - value: 66.03575000000001 - - type: recall_at_1000 - value: 83.94708333333334 - - type: recall_at_3 - value: 34.71941666666666 - - type: recall_at_5 - value: 39.46358333333333 - task: - type: Retrieval - - dataset: - config: default - name: MTEB ClimateFEVER - revision: None - split: test - type: climate-fever - metrics: - - type: map_at_1 - value: 9.024000000000001 - - type: map_at_10 - value: 15.644 - - type: map_at_100 - value: 17.154 - - type: map_at_1000 - value: 17.345 - - type: map_at_3 - value: 13.028 - - type: map_at_5 - value: 14.251 - - type: mrr_at_1 - value: 19.674 - - type: mrr_at_10 - value: 29.826999999999998 - - type: mrr_at_100 - value: 30.935000000000002 - - type: mrr_at_1000 - value: 30.987 - - type: mrr_at_3 - value: 26.645000000000003 - - type: mrr_at_5 - value: 28.29 - - type: ndcg_at_1 - value: 19.674 - - type: ndcg_at_10 - value: 22.545 - - type: ndcg_at_100 - value: 29.207 - - type: ndcg_at_1000 - value: 32.912 - - type: ndcg_at_3 - value: 17.952 - - type: ndcg_at_5 - value: 19.363 - - type: precision_at_1 - value: 19.674 - - type: precision_at_10 - value: 7.212000000000001 - - type: precision_at_100 - value: 1.435 - - type: precision_at_1000 - value: 0.212 - - type: precision_at_3 - value: 13.507 - - type: precision_at_5 - value: 10.397 - - type: recall_at_1 - value: 9.024000000000001 - - type: recall_at_10 - value: 28.077999999999996 - - type: recall_at_100 - value: 51.403 - - type: recall_at_1000 - value: 72.406 - - type: recall_at_3 - value: 16.768 - - type: recall_at_5 - value: 20.737 - task: - type: Retrieval - - dataset: - config: default - name: MTEB DBPedia - revision: None - split: test - type: dbpedia-entity - metrics: - - type: map_at_1 - value: 8.012 - - type: map_at_10 - value: 17.138 - - type: map_at_100 - value: 24.146 - - type: map_at_1000 - value: 25.622 - - type: map_at_3 - value: 12.552 - - type: map_at_5 - value: 14.435 - - type: mrr_at_1 - value: 62.25000000000001 - - type: mrr_at_10 - value: 71.186 - - type: mrr_at_100 - value: 71.504 - - type: mrr_at_1000 - value: 71.514 - - type: mrr_at_3 - value: 69.333 - - type: mrr_at_5 - value: 70.408 - - type: ndcg_at_1 - value: 49.75 - - type: ndcg_at_10 - value: 37.76 - - type: ndcg_at_100 - value: 42.071 - - type: ndcg_at_1000 - value: 49.309 - - type: ndcg_at_3 - value: 41.644 - - type: ndcg_at_5 - value: 39.812999999999995 - - type: precision_at_1 - value: 62.25000000000001 - - type: precision_at_10 - value: 30.15 - - type: precision_at_100 - value: 9.753 - - type: precision_at_1000 - value: 1.9189999999999998 - - type: precision_at_3 - value: 45.667 - - type: precision_at_5 - value: 39.15 - - type: recall_at_1 - value: 8.012 - - type: recall_at_10 - value: 22.599 - - type: recall_at_100 - value: 48.068 - - type: recall_at_1000 - value: 71.328 - - type: recall_at_3 - value: 14.043 - - type: recall_at_5 - value: 17.124 - task: - type: Retrieval - - dataset: - config: default - name: MTEB EmotionClassification - revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 - split: test - type: mteb/emotion - metrics: - - type: accuracy - value: 42.455 - - type: f1 - value: 37.59462649781862 - task: - type: Classification - - dataset: - config: default - name: MTEB FEVER - revision: None - split: test - type: fever - metrics: - - type: map_at_1 - value: 58.092 - - type: map_at_10 - value: 69.586 - - type: map_at_100 - value: 69.968 - - type: map_at_1000 - value: 69.982 - - type: map_at_3 - value: 67.48100000000001 - - type: map_at_5 - value: 68.915 - - type: mrr_at_1 - value: 62.166 - - type: mrr_at_10 - value: 73.588 - - type: mrr_at_100 - value: 73.86399999999999 - - type: mrr_at_1000 - value: 73.868 - - type: mrr_at_3 - value: 71.6 - - type: mrr_at_5 - value: 72.99 - - type: ndcg_at_1 - value: 62.166 - - type: ndcg_at_10 - value: 75.27199999999999 - - type: ndcg_at_100 - value: 76.816 - - type: ndcg_at_1000 - value: 77.09700000000001 - - type: ndcg_at_3 - value: 71.36 - - type: ndcg_at_5 - value: 73.785 - - type: precision_at_1 - value: 62.166 - - type: precision_at_10 - value: 9.716 - - type: precision_at_100 - value: 1.065 - - type: precision_at_1000 - value: 0.11 - - type: precision_at_3 - value: 28.278 - - type: precision_at_5 - value: 18.343999999999998 - - type: recall_at_1 - value: 58.092 - - type: recall_at_10 - value: 88.73400000000001 - - type: recall_at_100 - value: 95.195 - - type: recall_at_1000 - value: 97.04599999999999 - - type: recall_at_3 - value: 78.45 - - type: recall_at_5 - value: 84.316 - task: - type: Retrieval - - dataset: - config: default - name: MTEB FiQA2018 - revision: None - split: test - type: fiqa - metrics: - - type: map_at_1 - value: 16.649 - - type: map_at_10 - value: 26.457000000000004 - - type: map_at_100 - value: 28.169 - - type: map_at_1000 - value: 28.352 - - type: map_at_3 - value: 23.305 - - type: map_at_5 - value: 25.169000000000004 - - type: mrr_at_1 - value: 32.407000000000004 - - type: mrr_at_10 - value: 40.922 - - type: mrr_at_100 - value: 41.931000000000004 - - type: mrr_at_1000 - value: 41.983 - - type: mrr_at_3 - value: 38.786 - - type: mrr_at_5 - value: 40.205999999999996 - - type: ndcg_at_1 - value: 32.407000000000004 - - type: ndcg_at_10 - value: 33.314 - - type: ndcg_at_100 - value: 40.312 - - type: ndcg_at_1000 - value: 43.685 - - type: ndcg_at_3 - value: 30.391000000000002 - - type: ndcg_at_5 - value: 31.525 - - type: precision_at_1 - value: 32.407000000000004 - - type: precision_at_10 - value: 8.966000000000001 - - type: precision_at_100 - value: 1.6019999999999999 - - type: precision_at_1000 - value: 0.22200000000000003 - - type: precision_at_3 - value: 20.165 - - type: precision_at_5 - value: 14.722 - - type: recall_at_1 - value: 16.649 - - type: recall_at_10 - value: 39.117000000000004 - - type: recall_at_100 - value: 65.726 - - type: recall_at_1000 - value: 85.784 - - type: recall_at_3 - value: 27.914 - - type: recall_at_5 - value: 33.289 - task: - type: Retrieval - - dataset: - config: default - name: MTEB HotpotQA - revision: None - split: test - type: hotpotqa - metrics: - - type: map_at_1 - value: 36.253 - - type: map_at_10 - value: 56.16799999999999 - - type: map_at_100 - value: 57.06099999999999 - - type: map_at_1000 - value: 57.126 - - type: map_at_3 - value: 52.644999999999996 - - type: map_at_5 - value: 54.909 - - type: mrr_at_1 - value: 72.505 - - type: mrr_at_10 - value: 79.66 - - type: mrr_at_100 - value: 79.869 - - type: mrr_at_1000 - value: 79.88 - - type: mrr_at_3 - value: 78.411 - - type: mrr_at_5 - value: 79.19800000000001 - - type: ndcg_at_1 - value: 72.505 - - type: ndcg_at_10 - value: 65.094 - - type: ndcg_at_100 - value: 68.219 - - type: ndcg_at_1000 - value: 69.515 - - type: ndcg_at_3 - value: 59.99 - - type: ndcg_at_5 - value: 62.909000000000006 - - type: precision_at_1 - value: 72.505 - - type: precision_at_10 - value: 13.749 - - type: precision_at_100 - value: 1.619 - - type: precision_at_1000 - value: 0.179 - - type: precision_at_3 - value: 38.357 - - type: precision_at_5 - value: 25.313000000000002 - - type: recall_at_1 - value: 36.253 - - type: recall_at_10 - value: 68.744 - - type: recall_at_100 - value: 80.925 - - type: recall_at_1000 - value: 89.534 - - type: recall_at_3 - value: 57.535000000000004 - - type: recall_at_5 - value: 63.282000000000004 - task: - type: Retrieval - - dataset: - config: default - name: MTEB ImdbClassification - revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 - split: test - type: mteb/imdb - metrics: - - type: accuracy - value: 80.82239999999999 - - type: ap - value: 75.65895781725314 - - type: f1 - value: 80.75880969095746 - task: - type: Classification - - dataset: - config: default - name: MTEB MSMARCO - revision: None - split: dev - type: msmarco - metrics: - - type: map_at_1 - value: 21.624 - - type: map_at_10 - value: 34.075 - - type: map_at_100 - value: 35.229 - - type: map_at_1000 - value: 35.276999999999994 - - type: map_at_3 - value: 30.245 - - type: map_at_5 - value: 32.42 - - type: mrr_at_1 - value: 22.264 - - type: mrr_at_10 - value: 34.638000000000005 - - type: mrr_at_100 - value: 35.744 - - type: mrr_at_1000 - value: 35.787 - - type: mrr_at_3 - value: 30.891000000000002 - - type: mrr_at_5 - value: 33.042 - - type: ndcg_at_1 - value: 22.264 - - type: ndcg_at_10 - value: 40.991 - - type: ndcg_at_100 - value: 46.563 - - type: ndcg_at_1000 - value: 47.743 - - type: ndcg_at_3 - value: 33.198 - - type: ndcg_at_5 - value: 37.069 - - type: precision_at_1 - value: 22.264 - - type: precision_at_10 - value: 6.5089999999999995 - - type: precision_at_100 - value: 0.9299999999999999 - - type: precision_at_1000 - value: 0.10300000000000001 - - type: precision_at_3 - value: 14.216999999999999 - - type: precision_at_5 - value: 10.487 - - type: recall_at_1 - value: 21.624 - - type: recall_at_10 - value: 62.303 - - type: recall_at_100 - value: 88.124 - - type: recall_at_1000 - value: 97.08 - - type: recall_at_3 - value: 41.099999999999994 - - type: recall_at_5 - value: 50.381 - task: - type: Retrieval - - dataset: - config: en - name: MTEB MTOPDomainClassification (en) - revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf - split: test - type: mteb/mtop_domain - metrics: - - type: accuracy - value: 91.06703146374831 - - type: f1 - value: 90.86867815863172 - task: - type: Classification - - dataset: - config: de - name: MTEB MTOPDomainClassification (de) - revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf - split: test - type: mteb/mtop_domain - metrics: - - type: accuracy - value: 87.46970977740209 - - type: f1 - value: 86.36832872036588 - task: - type: Classification - - dataset: - config: es - name: MTEB MTOPDomainClassification (es) - revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf - split: test - type: mteb/mtop_domain - metrics: - - type: accuracy - value: 89.26951300867245 - - type: f1 - value: 88.93561193959502 - task: - type: Classification - - dataset: - config: fr - name: MTEB MTOPDomainClassification (fr) - revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf - split: test - type: mteb/mtop_domain - metrics: - - type: accuracy - value: 84.22799874725963 - - type: f1 - value: 84.30490069236556 - task: - type: Classification - - dataset: - config: hi - name: MTEB MTOPDomainClassification (hi) - revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf - split: test - type: mteb/mtop_domain - metrics: - - type: accuracy - value: 86.02007888131948 - - type: f1 - value: 85.39376041027991 - task: - type: Classification - - dataset: - config: th - name: MTEB MTOPDomainClassification (th) - revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf - split: test - type: mteb/mtop_domain - metrics: - - type: accuracy - value: 85.34900542495481 - - type: f1 - value: 85.39859673336713 - task: - type: Classification - - dataset: - config: en - name: MTEB MTOPIntentClassification (en) - revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba - split: test - type: mteb/mtop_intent - metrics: - - type: accuracy - value: 71.078431372549 - - type: f1 - value: 53.45071102002276 - task: - type: Classification - - dataset: - config: de - name: MTEB MTOPIntentClassification (de) - revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba - split: test - type: mteb/mtop_intent - metrics: - - type: accuracy - value: 65.85798816568047 - - type: f1 - value: 46.53112748993529 - task: - type: Classification - - dataset: - config: es - name: MTEB MTOPIntentClassification (es) - revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba - split: test - type: mteb/mtop_intent - metrics: - - type: accuracy - value: 67.96864576384256 - - type: f1 - value: 45.966703022829506 - task: - type: Classification - - dataset: - config: fr - name: MTEB MTOPIntentClassification (fr) - revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba - split: test - type: mteb/mtop_intent - metrics: - - type: accuracy - value: 61.31537738803633 - - type: f1 - value: 45.52601712835461 - task: - type: Classification - - dataset: - config: hi - name: MTEB MTOPIntentClassification (hi) - revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba - split: test - type: mteb/mtop_intent - metrics: - - type: accuracy - value: 66.29616349946218 - - type: f1 - value: 47.24166485726613 - task: - type: Classification - - dataset: - config: th - name: MTEB MTOPIntentClassification (th) - revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba - split: test - type: mteb/mtop_intent - metrics: - - type: accuracy - value: 67.51537070524412 - - type: f1 - value: 49.463476319014276 - task: - type: Classification - - dataset: - config: af - name: MTEB MassiveIntentClassification (af) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 57.06792199058508 - - type: f1 - value: 54.094921857502285 - task: - type: Classification - - dataset: - config: am - name: MTEB MassiveIntentClassification (am) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 51.960322797579025 - - type: f1 - value: 48.547371223370945 - task: - type: Classification - - dataset: - config: ar - name: MTEB MassiveIntentClassification (ar) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 54.425016812373904 - - type: f1 - value: 50.47069202054312 - task: - type: Classification - - dataset: - config: az - name: MTEB MassiveIntentClassification (az) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 59.798251513113655 - - type: f1 - value: 57.05013069086648 - task: - type: Classification - - dataset: - config: bn - name: MTEB MassiveIntentClassification (bn) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 59.37794216543376 - - type: f1 - value: 56.3607992649805 - task: - type: Classification - - dataset: - config: cy - name: MTEB MassiveIntentClassification (cy) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 46.56018829858777 - - type: f1 - value: 43.87319715715134 - task: - type: Classification - - dataset: - config: da - name: MTEB MassiveIntentClassification (da) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 62.9724277067922 - - type: f1 - value: 59.36480066245562 - task: - type: Classification - - dataset: - config: de - name: MTEB MassiveIntentClassification (de) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 62.72696704774715 - - type: f1 - value: 59.143595966615855 - task: - type: Classification - - dataset: - config: el - name: MTEB MassiveIntentClassification (el) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 61.5971755211836 - - type: f1 - value: 59.169445724946726 - task: - type: Classification - - dataset: - config: en - name: MTEB MassiveIntentClassification (en) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 70.29589778076665 - - type: f1 - value: 67.7577001808977 - task: - type: Classification - - dataset: - config: es - name: MTEB MassiveIntentClassification (es) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 66.31136516476126 - - type: f1 - value: 64.52032955983242 - task: - type: Classification - - dataset: - config: fa - name: MTEB MassiveIntentClassification (fa) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 65.54472091459314 - - type: f1 - value: 61.47903120066317 - task: - type: Classification - - dataset: - config: fi - name: MTEB MassiveIntentClassification (fi) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 61.45595158036314 - - type: f1 - value: 58.0891846024637 - task: - type: Classification - - dataset: - config: fr - name: MTEB MassiveIntentClassification (fr) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 65.47074646940149 - - type: f1 - value: 62.84830858877575 - task: - type: Classification - - dataset: - config: he - name: MTEB MassiveIntentClassification (he) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 58.046402151983855 - - type: f1 - value: 55.269074430533195 - task: - type: Classification - - dataset: - config: hi - name: MTEB MassiveIntentClassification (hi) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 64.06523201075991 - - type: f1 - value: 61.35339643021369 - task: - type: Classification - - dataset: - config: hu - name: MTEB MassiveIntentClassification (hu) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 60.954942837928726 - - type: f1 - value: 57.07035922704846 - task: - type: Classification - - dataset: - config: hy - name: MTEB MassiveIntentClassification (hy) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 57.404169468728995 - - type: f1 - value: 53.94259011839138 - task: - type: Classification - - dataset: - config: id - name: MTEB MassiveIntentClassification (id) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 64.16610625420309 - - type: f1 - value: 61.337103431499365 - task: - type: Classification - - dataset: - config: is - name: MTEB MassiveIntentClassification (is) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 52.262945527908535 - - type: f1 - value: 49.7610691598921 - task: - type: Classification - - dataset: - config: it - name: MTEB MassiveIntentClassification (it) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 65.54472091459314 - - type: f1 - value: 63.469099018440154 - task: - type: Classification - - dataset: - config: ja - name: MTEB MassiveIntentClassification (ja) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 68.22797579018157 - - type: f1 - value: 64.89098471083001 - task: - type: Classification - - dataset: - config: jv - name: MTEB MassiveIntentClassification (jv) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 50.847343644922674 - - type: f1 - value: 47.8536963168393 - task: - type: Classification - - dataset: - config: ka - name: MTEB MassiveIntentClassification (ka) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 48.45326160053799 - - type: f1 - value: 46.370078045805556 - task: - type: Classification - - dataset: - config: km - name: MTEB MassiveIntentClassification (km) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 42.83120376597175 - - type: f1 - value: 39.68948521599982 - task: - type: Classification - - dataset: - config: kn - name: MTEB MassiveIntentClassification (kn) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 57.5084061869536 - - type: f1 - value: 53.961876160401545 - task: - type: Classification - - dataset: - config: ko - name: MTEB MassiveIntentClassification (ko) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 63.7895090786819 - - type: f1 - value: 61.134223684676 - task: - type: Classification - - dataset: - config: lv - name: MTEB MassiveIntentClassification (lv) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 54.98991257565569 - - type: f1 - value: 52.579862862826296 - task: - type: Classification - - dataset: - config: ml - name: MTEB MassiveIntentClassification (ml) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 61.90316072629456 - - type: f1 - value: 58.203024538290336 - task: - type: Classification - - dataset: - config: mn - name: MTEB MassiveIntentClassification (mn) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 57.09818426361802 - - type: f1 - value: 54.22718458445455 - task: - type: Classification - - dataset: - config: ms - name: MTEB MassiveIntentClassification (ms) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 58.991257565568255 - - type: f1 - value: 55.84892781767421 - task: - type: Classification - - dataset: - config: my - name: MTEB MassiveIntentClassification (my) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 55.901143241425686 - - type: f1 - value: 52.25264332199797 - task: - type: Classification - - dataset: - config: nb - name: MTEB MassiveIntentClassification (nb) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 61.96368527236047 - - type: f1 - value: 58.927243876153454 - task: - type: Classification - - dataset: - config: nl - name: MTEB MassiveIntentClassification (nl) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 65.64223268325489 - - type: f1 - value: 62.340453718379706 - task: - type: Classification - - dataset: - config: pl - name: MTEB MassiveIntentClassification (pl) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 64.52589105581708 - - type: f1 - value: 61.661113187022174 - task: - type: Classification - - dataset: - config: pt - name: MTEB MassiveIntentClassification (pt) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 66.84599865501009 - - type: f1 - value: 64.59342572873005 - task: - type: Classification - - dataset: - config: ro - name: MTEB MassiveIntentClassification (ro) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 60.81035642232684 - - type: f1 - value: 57.5169089806797 - task: - type: Classification - - dataset: - config: ru - name: MTEB MassiveIntentClassification (ru) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 58.652238071815056 - - type: f1 - value: 53.22732406426353 - - type: f1_weighted - value: 57.585586737209546 - - type: main_score - value: 58.652238071815056 - task: - type: Classification - - dataset: - config: sl - name: MTEB MassiveIntentClassification (sl) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 56.51647612642906 - - type: f1 - value: 54.33154780100043 - task: - type: Classification - - dataset: - config: sq - name: MTEB MassiveIntentClassification (sq) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 57.985877605917956 - - type: f1 - value: 54.46187524463802 - task: - type: Classification - - dataset: - config: sv - name: MTEB MassiveIntentClassification (sv) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 65.03026227303296 - - type: f1 - value: 62.34377392877748 - task: - type: Classification - - dataset: - config: sw - name: MTEB MassiveIntentClassification (sw) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 53.567585743106925 - - type: f1 - value: 50.73770655983206 - task: - type: Classification - - dataset: - config: ta - name: MTEB MassiveIntentClassification (ta) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 57.2595830531271 - - type: f1 - value: 53.657327291708626 - task: - type: Classification - - dataset: - config: te - name: MTEB MassiveIntentClassification (te) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 57.82784129119032 - - type: f1 - value: 54.82518072665301 - task: - type: Classification - - dataset: - config: th - name: MTEB MassiveIntentClassification (th) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 64.06859448554137 - - type: f1 - value: 63.00185280500495 - task: - type: Classification - - dataset: - config: tl - name: MTEB MassiveIntentClassification (tl) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 58.91055817081371 - - type: f1 - value: 55.54116301224262 - task: - type: Classification - - dataset: - config: tr - name: MTEB MassiveIntentClassification (tr) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 63.54404841963686 - - type: f1 - value: 59.57650946030184 - task: - type: Classification - - dataset: - config: ur - name: MTEB MassiveIntentClassification (ur) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 59.27706792199059 - - type: f1 - value: 56.50010066083435 - task: - type: Classification - - dataset: - config: vi - name: MTEB MassiveIntentClassification (vi) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 64.0719569603228 - - type: f1 - value: 61.817075925647956 - task: - type: Classification - - dataset: - config: zh-CN - name: MTEB MassiveIntentClassification (zh-CN) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 68.23806321452591 - - type: f1 - value: 65.24917026029749 - task: - type: Classification - - dataset: - config: zh-TW - name: MTEB MassiveIntentClassification (zh-TW) - revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 - split: test - type: mteb/amazon_massive_intent - metrics: - - type: accuracy - value: 62.53530598520511 - - type: f1 - value: 61.71131132295768 - task: - type: Classification - - dataset: - config: af - name: MTEB MassiveScenarioClassification (af) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 63.04303967720243 - - type: f1 - value: 60.3950085685985 - task: - type: Classification - - dataset: - config: am - name: MTEB MassiveScenarioClassification (am) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 56.83591123066578 - - type: f1 - value: 54.95059828830849 - task: - type: Classification - - dataset: - config: ar - name: MTEB MassiveScenarioClassification (ar) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 59.62340282447881 - - type: f1 - value: 59.525159996498225 - task: - type: Classification - - dataset: - config: az - name: MTEB MassiveScenarioClassification (az) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 60.85406859448555 - - type: f1 - value: 59.129299095681276 - task: - type: Classification - - dataset: - config: bn - name: MTEB MassiveScenarioClassification (bn) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 62.76731674512441 - - type: f1 - value: 61.159560612627715 - task: - type: Classification - - dataset: - config: cy - name: MTEB MassiveScenarioClassification (cy) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 50.181573638197705 - - type: f1 - value: 46.98422176289957 - task: - type: Classification - - dataset: - config: da - name: MTEB MassiveScenarioClassification (da) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 68.92737054472092 - - type: f1 - value: 67.69135611952979 - task: - type: Classification - - dataset: - config: de - name: MTEB MassiveScenarioClassification (de) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 69.18964357767318 - - type: f1 - value: 68.46106138186214 - task: - type: Classification - - dataset: - config: el - name: MTEB MassiveScenarioClassification (el) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 67.0712844653665 - - type: f1 - value: 66.75545422473901 - task: - type: Classification - - dataset: - config: en - name: MTEB MassiveScenarioClassification (en) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 74.4754539340955 - - type: f1 - value: 74.38427146553252 - task: - type: Classification - - dataset: - config: es - name: MTEB MassiveScenarioClassification (es) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 69.82515131136518 - - type: f1 - value: 69.63516462173847 - task: - type: Classification - - dataset: - config: fa - name: MTEB MassiveScenarioClassification (fa) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 68.70880968392737 - - type: f1 - value: 67.45420662567926 - task: - type: Classification - - dataset: - config: fi - name: MTEB MassiveScenarioClassification (fi) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 65.95494283792871 - - type: f1 - value: 65.06191009049222 - task: - type: Classification - - dataset: - config: fr - name: MTEB MassiveScenarioClassification (fr) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 68.75924680564896 - - type: f1 - value: 68.30833379585945 - task: - type: Classification - - dataset: - config: he - name: MTEB MassiveScenarioClassification (he) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 63.806321452589096 - - type: f1 - value: 63.273048243765054 - task: - type: Classification - - dataset: - config: hi - name: MTEB MassiveScenarioClassification (hi) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 67.68997982515133 - - type: f1 - value: 66.54703855381324 - task: - type: Classification - - dataset: - config: hu - name: MTEB MassiveScenarioClassification (hu) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 66.46940147948891 - - type: f1 - value: 65.91017343463396 - task: - type: Classification - - dataset: - config: hy - name: MTEB MassiveScenarioClassification (hy) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 59.49899125756556 - - type: f1 - value: 57.90333469917769 - task: - type: Classification - - dataset: - config: id - name: MTEB MassiveScenarioClassification (id) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 67.9219905850706 - - type: f1 - value: 67.23169403762938 - task: - type: Classification - - dataset: - config: is - name: MTEB MassiveScenarioClassification (is) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 56.486213853396094 - - type: f1 - value: 54.85282355583758 - task: - type: Classification - - dataset: - config: it - name: MTEB MassiveScenarioClassification (it) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 69.04169468728985 - - type: f1 - value: 68.83833333320462 - task: - type: Classification - - dataset: - config: ja - name: MTEB MassiveScenarioClassification (ja) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 73.88702084734365 - - type: f1 - value: 74.04474735232299 - task: - type: Classification - - dataset: - config: jv - name: MTEB MassiveScenarioClassification (jv) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 56.63416274377943 - - type: f1 - value: 55.11332211687954 - task: - type: Classification - - dataset: - config: ka - name: MTEB MassiveScenarioClassification (ka) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 52.23604572965702 - - type: f1 - value: 50.86529813991055 - task: - type: Classification - - dataset: - config: km - name: MTEB MassiveScenarioClassification (km) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 46.62407531943511 - - type: f1 - value: 43.63485467164535 - task: - type: Classification - - dataset: - config: kn - name: MTEB MassiveScenarioClassification (kn) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 59.15601882985878 - - type: f1 - value: 57.522837510959924 - task: - type: Classification - - dataset: - config: ko - name: MTEB MassiveScenarioClassification (ko) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 69.84532616005382 - - type: f1 - value: 69.60021127179697 - task: - type: Classification - - dataset: - config: lv - name: MTEB MassiveScenarioClassification (lv) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 56.65770006724949 - - type: f1 - value: 55.84219135523227 - task: - type: Classification - - dataset: - config: ml - name: MTEB MassiveScenarioClassification (ml) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 66.53665097511768 - - type: f1 - value: 65.09087787792639 - task: - type: Classification - - dataset: - config: mn - name: MTEB MassiveScenarioClassification (mn) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 59.31405514458642 - - type: f1 - value: 58.06135303831491 - task: - type: Classification - - dataset: - config: ms - name: MTEB MassiveScenarioClassification (ms) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 64.88231338264964 - - type: f1 - value: 62.751099407787926 - task: - type: Classification - - dataset: - config: my - name: MTEB MassiveScenarioClassification (my) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 58.86012104909213 - - type: f1 - value: 56.29118323058282 - task: - type: Classification - - dataset: - config: nb - name: MTEB MassiveScenarioClassification (nb) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 67.37390719569602 - - type: f1 - value: 66.27922244885102 - task: - type: Classification - - dataset: - config: nl - name: MTEB MassiveScenarioClassification (nl) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 70.8675184936113 - - type: f1 - value: 70.22146529932019 - task: - type: Classification - - dataset: - config: pl - name: MTEB MassiveScenarioClassification (pl) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 68.2212508406187 - - type: f1 - value: 67.77454802056282 - task: - type: Classification - - dataset: - config: pt - name: MTEB MassiveScenarioClassification (pt) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 68.18090114324143 - - type: f1 - value: 68.03737625431621 - task: - type: Classification - - dataset: - config: ro - name: MTEB MassiveScenarioClassification (ro) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 64.65030262273034 - - type: f1 - value: 63.792945486912856 - task: - type: Classification - - dataset: - config: ru - name: MTEB MassiveScenarioClassification (ru) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 63.772749631087066 - - type: f1 - value: 63.4539101720024 - - type: f1_weighted - value: 62.778603897469566 - - type: main_score - value: 63.772749631087066 - task: - type: Classification - - dataset: - config: sl - name: MTEB MassiveScenarioClassification (sl) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 60.17821116341627 - - type: f1 - value: 59.3935969827171 - task: - type: Classification - - dataset: - config: sq - name: MTEB MassiveScenarioClassification (sq) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 62.86146603900471 - - type: f1 - value: 60.133692735032376 - task: - type: Classification - - dataset: - config: sv - name: MTEB MassiveScenarioClassification (sv) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 70.89441829186282 - - type: f1 - value: 70.03064076194089 - task: - type: Classification - - dataset: - config: sw - name: MTEB MassiveScenarioClassification (sw) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 58.15063887020847 - - type: f1 - value: 56.23326278499678 - task: - type: Classification - - dataset: - config: ta - name: MTEB MassiveScenarioClassification (ta) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 59.43846671149966 - - type: f1 - value: 57.70440450281974 - task: - type: Classification - - dataset: - config: te - name: MTEB MassiveScenarioClassification (te) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 60.8507061197041 - - type: f1 - value: 59.22916396061171 - task: - type: Classification - - dataset: - config: th - name: MTEB MassiveScenarioClassification (th) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 70.65568258238063 - - type: f1 - value: 69.90736239440633 - task: - type: Classification - - dataset: - config: tl - name: MTEB MassiveScenarioClassification (tl) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 60.8843308675185 - - type: f1 - value: 59.30332663713599 - task: - type: Classification - - dataset: - config: tr - name: MTEB MassiveScenarioClassification (tr) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 68.05312710154674 - - type: f1 - value: 67.44024062594775 - task: - type: Classification - - dataset: - config: ur - name: MTEB MassiveScenarioClassification (ur) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 62.111634162743776 - - type: f1 - value: 60.89083013084519 - task: - type: Classification - - dataset: - config: vi - name: MTEB MassiveScenarioClassification (vi) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 67.44115669132482 - - type: f1 - value: 67.92227541674552 - task: - type: Classification - - dataset: - config: zh-CN - name: MTEB MassiveScenarioClassification (zh-CN) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 74.4687289845326 - - type: f1 - value: 74.16376793486025 - task: - type: Classification - - dataset: - config: zh-TW - name: MTEB MassiveScenarioClassification (zh-TW) - revision: 7d571f92784cd94a019292a1f45445077d0ef634 - split: test - type: mteb/amazon_massive_scenario - metrics: - - type: accuracy - value: 68.31876260928043 - - type: f1 - value: 68.5246745215607 - task: - type: Classification - - dataset: - config: default - name: MTEB MedrxivClusteringP2P - revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 - split: test - type: mteb/medrxiv-clustering-p2p - metrics: - - type: v_measure - value: 30.90431696479766 - task: - type: Clustering - - dataset: - config: default - name: MTEB MedrxivClusteringS2S - revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 - split: test - type: mteb/medrxiv-clustering-s2s - metrics: - - type: v_measure - value: 27.259158476693774 - task: - type: Clustering - - dataset: - config: default - name: MTEB MindSmallReranking - revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69 - split: test - type: mteb/mind_small - metrics: - - type: map - value: 30.28445330838555 - - type: mrr - value: 31.15758529581164 - task: - type: Reranking - - dataset: - config: default - name: MTEB NFCorpus - revision: None - split: test - type: nfcorpus - metrics: - - type: map_at_1 - value: 5.353 - - type: map_at_10 - value: 11.565 - - type: map_at_100 - value: 14.097000000000001 - - type: map_at_1000 - value: 15.354999999999999 - - type: map_at_3 - value: 8.749 - - type: map_at_5 - value: 9.974 - - type: mrr_at_1 - value: 42.105 - - type: mrr_at_10 - value: 50.589 - - type: mrr_at_100 - value: 51.187000000000005 - - type: mrr_at_1000 - value: 51.233 - - type: mrr_at_3 - value: 48.246 - - type: mrr_at_5 - value: 49.546 - - type: ndcg_at_1 - value: 40.402 - - type: ndcg_at_10 - value: 31.009999999999998 - - type: ndcg_at_100 - value: 28.026 - - type: ndcg_at_1000 - value: 36.905 - - type: ndcg_at_3 - value: 35.983 - - type: ndcg_at_5 - value: 33.764 - - type: precision_at_1 - value: 42.105 - - type: precision_at_10 - value: 22.786 - - type: precision_at_100 - value: 6.916 - - type: precision_at_1000 - value: 1.981 - - type: precision_at_3 - value: 33.333 - - type: precision_at_5 - value: 28.731 - - type: recall_at_1 - value: 5.353 - - type: recall_at_10 - value: 15.039 - - type: recall_at_100 - value: 27.348 - - type: recall_at_1000 - value: 59.453 - - type: recall_at_3 - value: 9.792 - - type: recall_at_5 - value: 11.882 - task: - type: Retrieval - - dataset: - config: default - name: MTEB NQ - revision: None - split: test - type: nq - metrics: - - type: map_at_1 - value: 33.852 - - type: map_at_10 - value: 48.924 - - type: map_at_100 - value: 49.854 - - type: map_at_1000 - value: 49.886 - - type: map_at_3 - value: 44.9 - - type: map_at_5 - value: 47.387 - - type: mrr_at_1 - value: 38.035999999999994 - - type: mrr_at_10 - value: 51.644 - - type: mrr_at_100 - value: 52.339 - - type: mrr_at_1000 - value: 52.35999999999999 - - type: mrr_at_3 - value: 48.421 - - type: mrr_at_5 - value: 50.468999999999994 - - type: ndcg_at_1 - value: 38.007000000000005 - - type: ndcg_at_10 - value: 56.293000000000006 - - type: ndcg_at_100 - value: 60.167 - - type: ndcg_at_1000 - value: 60.916000000000004 - - type: ndcg_at_3 - value: 48.903999999999996 - - type: ndcg_at_5 - value: 52.978 - - type: precision_at_1 - value: 38.007000000000005 - - type: precision_at_10 - value: 9.041 - - type: precision_at_100 - value: 1.1199999999999999 - - type: precision_at_1000 - value: 0.11900000000000001 - - type: precision_at_3 - value: 22.084 - - type: precision_at_5 - value: 15.608 - - type: recall_at_1 - value: 33.852 - - type: recall_at_10 - value: 75.893 - - type: recall_at_100 - value: 92.589 - - type: recall_at_1000 - value: 98.153 - - type: recall_at_3 - value: 56.969 - - type: recall_at_5 - value: 66.283 - task: - type: Retrieval - - dataset: - config: default - name: MTEB QuoraRetrieval - revision: None - split: test - type: quora - metrics: - - type: map_at_1 - value: 69.174 - - type: map_at_10 - value: 82.891 - - type: map_at_100 - value: 83.545 - - type: map_at_1000 - value: 83.56700000000001 - - type: map_at_3 - value: 79.944 - - type: map_at_5 - value: 81.812 - - type: mrr_at_1 - value: 79.67999999999999 - - type: mrr_at_10 - value: 86.279 - - type: mrr_at_100 - value: 86.39 - - type: mrr_at_1000 - value: 86.392 - - type: mrr_at_3 - value: 85.21 - - type: mrr_at_5 - value: 85.92999999999999 - - type: ndcg_at_1 - value: 79.69000000000001 - - type: ndcg_at_10 - value: 86.929 - - type: ndcg_at_100 - value: 88.266 - - type: ndcg_at_1000 - value: 88.428 - - type: ndcg_at_3 - value: 83.899 - - type: ndcg_at_5 - value: 85.56700000000001 - - type: precision_at_1 - value: 79.69000000000001 - - type: precision_at_10 - value: 13.161000000000001 - - type: precision_at_100 - value: 1.513 - - type: precision_at_1000 - value: 0.156 - - type: precision_at_3 - value: 36.603 - - type: precision_at_5 - value: 24.138 - - type: recall_at_1 - value: 69.174 - - type: recall_at_10 - value: 94.529 - - type: recall_at_100 - value: 99.15 - - type: recall_at_1000 - value: 99.925 - - type: recall_at_3 - value: 85.86200000000001 - - type: recall_at_5 - value: 90.501 - task: - type: Retrieval - - dataset: - config: default - name: MTEB RedditClustering - revision: 24640382cdbf8abc73003fb0fa6d111a705499eb - split: test - type: mteb/reddit-clustering - metrics: - - type: v_measure - value: 39.13064340585255 - task: - type: Clustering - - dataset: - config: default - name: MTEB RedditClusteringP2P - revision: 282350215ef01743dc01b456c7f5241fa8937f16 - split: test - type: mteb/reddit-clustering-p2p - metrics: - - type: v_measure - value: 58.97884249325877 - task: - type: Clustering - - dataset: - config: default - name: MTEB SCIDOCS - revision: None - split: test - type: scidocs - metrics: - - type: map_at_1 - value: 3.4680000000000004 - - type: map_at_10 - value: 7.865 - - type: map_at_100 - value: 9.332 - - type: map_at_1000 - value: 9.587 - - type: map_at_3 - value: 5.800000000000001 - - type: map_at_5 - value: 6.8790000000000004 - - type: mrr_at_1 - value: 17.0 - - type: mrr_at_10 - value: 25.629 - - type: mrr_at_100 - value: 26.806 - - type: mrr_at_1000 - value: 26.889000000000003 - - type: mrr_at_3 - value: 22.8 - - type: mrr_at_5 - value: 24.26 - - type: ndcg_at_1 - value: 17.0 - - type: ndcg_at_10 - value: 13.895 - - type: ndcg_at_100 - value: 20.491999999999997 - - type: ndcg_at_1000 - value: 25.759999999999998 - - type: ndcg_at_3 - value: 13.347999999999999 - - type: ndcg_at_5 - value: 11.61 - - type: precision_at_1 - value: 17.0 - - type: precision_at_10 - value: 7.090000000000001 - - type: precision_at_100 - value: 1.669 - - type: precision_at_1000 - value: 0.294 - - type: precision_at_3 - value: 12.3 - - type: precision_at_5 - value: 10.02 - - type: recall_at_1 - value: 3.4680000000000004 - - type: recall_at_10 - value: 14.363000000000001 - - type: recall_at_100 - value: 33.875 - - type: recall_at_1000 - value: 59.711999999999996 - - type: recall_at_3 - value: 7.483 - - type: recall_at_5 - value: 10.173 - task: - type: Retrieval - - dataset: - config: default - name: MTEB SICK-R - revision: a6ea5a8cab320b040a23452cc28066d9beae2cee - split: test - type: mteb/sickr-sts - metrics: - - type: cos_sim_pearson - value: 83.04084311714061 - - type: cos_sim_spearman - value: 77.51342467443078 - - type: euclidean_pearson - value: 80.0321166028479 - - type: euclidean_spearman - value: 77.29249114733226 - - type: manhattan_pearson - value: 80.03105964262431 - - type: manhattan_spearman - value: 77.22373689514794 - task: - type: STS - - dataset: - config: default - name: MTEB STS12 - revision: a0d554a64d88156834ff5ae9920b964011b16384 - split: test - type: mteb/sts12-sts - metrics: - - type: cos_sim_pearson - value: 84.1680158034387 - - type: cos_sim_spearman - value: 76.55983344071117 - - type: euclidean_pearson - value: 79.75266678300143 - - type: euclidean_spearman - value: 75.34516823467025 - - type: manhattan_pearson - value: 79.75959151517357 - - type: manhattan_spearman - value: 75.42330344141912 - task: - type: STS - - dataset: - config: default - name: MTEB STS13 - revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca - split: test - type: mteb/sts13-sts - metrics: - - type: cos_sim_pearson - value: 76.48898993209346 - - type: cos_sim_spearman - value: 76.96954120323366 - - type: euclidean_pearson - value: 76.94139109279668 - - type: euclidean_spearman - value: 76.85860283201711 - - type: manhattan_pearson - value: 76.6944095091912 - - type: manhattan_spearman - value: 76.61096912972553 - task: - type: STS - - dataset: - config: default - name: MTEB STS14 - revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375 - split: test - type: mteb/sts14-sts - metrics: - - type: cos_sim_pearson - value: 77.85082366246944 - - type: cos_sim_spearman - value: 75.52053350101731 - - type: euclidean_pearson - value: 77.1165845070926 - - type: euclidean_spearman - value: 75.31216065884388 - - type: manhattan_pearson - value: 77.06193941833494 - - type: manhattan_spearman - value: 75.31003701700112 - task: - type: STS - - dataset: - config: default - name: MTEB STS15 - revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3 - split: test - type: mteb/sts15-sts - metrics: - - type: cos_sim_pearson - value: 86.36305246526497 - - type: cos_sim_spearman - value: 87.11704613927415 - - type: euclidean_pearson - value: 86.04199125810939 - - type: euclidean_spearman - value: 86.51117572414263 - - type: manhattan_pearson - value: 86.0805106816633 - - type: manhattan_spearman - value: 86.52798366512229 - task: - type: STS - - dataset: - config: default - name: MTEB STS16 - revision: 4d8694f8f0e0100860b497b999b3dbed754a0513 - split: test - type: mteb/sts16-sts - metrics: - - type: cos_sim_pearson - value: 82.18536255599724 - - type: cos_sim_spearman - value: 83.63377151025418 - - type: euclidean_pearson - value: 83.24657467993141 - - type: euclidean_spearman - value: 84.02751481993825 - - type: manhattan_pearson - value: 83.11941806582371 - - type: manhattan_spearman - value: 83.84251281019304 - task: - type: STS - - dataset: - config: ko-ko - name: MTEB STS17 (ko-ko) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 78.95816528475514 - - type: cos_sim_spearman - value: 78.86607380120462 - - type: euclidean_pearson - value: 78.51268699230545 - - type: euclidean_spearman - value: 79.11649316502229 - - type: manhattan_pearson - value: 78.32367302808157 - - type: manhattan_spearman - value: 78.90277699624637 - task: - type: STS - - dataset: - config: ar-ar - name: MTEB STS17 (ar-ar) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 72.89126914997624 - - type: cos_sim_spearman - value: 73.0296921832678 - - type: euclidean_pearson - value: 71.50385903677738 - - type: euclidean_spearman - value: 73.13368899716289 - - type: manhattan_pearson - value: 71.47421463379519 - - type: manhattan_spearman - value: 73.03383242946575 - task: - type: STS - - dataset: - config: en-ar - name: MTEB STS17 (en-ar) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 59.22923684492637 - - type: cos_sim_spearman - value: 57.41013211368396 - - type: euclidean_pearson - value: 61.21107388080905 - - type: euclidean_spearman - value: 60.07620768697254 - - type: manhattan_pearson - value: 59.60157142786555 - - type: manhattan_spearman - value: 59.14069604103739 - task: - type: STS - - dataset: - config: en-de - name: MTEB STS17 (en-de) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 76.24345978774299 - - type: cos_sim_spearman - value: 77.24225743830719 - - type: euclidean_pearson - value: 76.66226095469165 - - type: euclidean_spearman - value: 77.60708820493146 - - type: manhattan_pearson - value: 76.05303324760429 - - type: manhattan_spearman - value: 76.96353149912348 - task: - type: STS - - dataset: - config: en-en - name: MTEB STS17 (en-en) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 85.50879160160852 - - type: cos_sim_spearman - value: 86.43594662965224 - - type: euclidean_pearson - value: 86.06846012826577 - - type: euclidean_spearman - value: 86.02041395794136 - - type: manhattan_pearson - value: 86.10916255616904 - - type: manhattan_spearman - value: 86.07346068198953 - task: - type: STS - - dataset: - config: en-tr - name: MTEB STS17 (en-tr) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 58.39803698977196 - - type: cos_sim_spearman - value: 55.96910950423142 - - type: euclidean_pearson - value: 58.17941175613059 - - type: euclidean_spearman - value: 55.03019330522745 - - type: manhattan_pearson - value: 57.333358138183286 - - type: manhattan_spearman - value: 54.04614023149965 - task: - type: STS - - dataset: - config: es-en - name: MTEB STS17 (es-en) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 70.98304089637197 - - type: cos_sim_spearman - value: 72.44071656215888 - - type: euclidean_pearson - value: 72.19224359033983 - - type: euclidean_spearman - value: 73.89871188913025 - - type: manhattan_pearson - value: 71.21098311547406 - - type: manhattan_spearman - value: 72.93405764824821 - task: - type: STS - - dataset: - config: es-es - name: MTEB STS17 (es-es) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 85.99792397466308 - - type: cos_sim_spearman - value: 84.83824377879495 - - type: euclidean_pearson - value: 85.70043288694438 - - type: euclidean_spearman - value: 84.70627558703686 - - type: manhattan_pearson - value: 85.89570850150801 - - type: manhattan_spearman - value: 84.95806105313007 - task: - type: STS - - dataset: - config: fr-en - name: MTEB STS17 (fr-en) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 72.21850322994712 - - type: cos_sim_spearman - value: 72.28669398117248 - - type: euclidean_pearson - value: 73.40082510412948 - - type: euclidean_spearman - value: 73.0326539281865 - - type: manhattan_pearson - value: 71.8659633964841 - - type: manhattan_spearman - value: 71.57817425823303 - task: - type: STS - - dataset: - config: it-en - name: MTEB STS17 (it-en) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 75.80921368595645 - - type: cos_sim_spearman - value: 77.33209091229315 - - type: euclidean_pearson - value: 76.53159540154829 - - type: euclidean_spearman - value: 78.17960842810093 - - type: manhattan_pearson - value: 76.13530186637601 - - type: manhattan_spearman - value: 78.00701437666875 - task: - type: STS - - dataset: - config: nl-en - name: MTEB STS17 (nl-en) - revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d - split: test - type: mteb/sts17-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 74.74980608267349 - - type: cos_sim_spearman - value: 75.37597374318821 - - type: euclidean_pearson - value: 74.90506081911661 - - type: euclidean_spearman - value: 75.30151613124521 - - type: manhattan_pearson - value: 74.62642745918002 - - type: manhattan_spearman - value: 75.18619716592303 - task: - type: STS - - dataset: - config: en - name: MTEB STS22 (en) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 59.632662289205584 - - type: cos_sim_spearman - value: 60.938543391610914 - - type: euclidean_pearson - value: 62.113200529767056 - - type: euclidean_spearman - value: 61.410312633261164 - - type: manhattan_pearson - value: 61.75494698945686 - - type: manhattan_spearman - value: 60.92726195322362 - task: - type: STS - - dataset: - config: de - name: MTEB STS22 (de) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 45.283470551557244 - - type: cos_sim_spearman - value: 53.44833015864201 - - type: euclidean_pearson - value: 41.17892011120893 - - type: euclidean_spearman - value: 53.81441383126767 - - type: manhattan_pearson - value: 41.17482200420659 - - type: manhattan_spearman - value: 53.82180269276363 - task: - type: STS - - dataset: - config: es - name: MTEB STS22 (es) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 60.5069165306236 - - type: cos_sim_spearman - value: 66.87803259033826 - - type: euclidean_pearson - value: 63.5428979418236 - - type: euclidean_spearman - value: 66.9293576586897 - - type: manhattan_pearson - value: 63.59789526178922 - - type: manhattan_spearman - value: 66.86555009875066 - task: - type: STS - - dataset: - config: pl - name: MTEB STS22 (pl) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 28.23026196280264 - - type: cos_sim_spearman - value: 35.79397812652861 - - type: euclidean_pearson - value: 17.828102102767353 - - type: euclidean_spearman - value: 35.721501145568894 - - type: manhattan_pearson - value: 17.77134274219677 - - type: manhattan_spearman - value: 35.98107902846267 - task: - type: STS - - dataset: - config: tr - name: MTEB STS22 (tr) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 56.51946541393812 - - type: cos_sim_spearman - value: 63.714686006214485 - - type: euclidean_pearson - value: 58.32104651305898 - - type: euclidean_spearman - value: 62.237110895702216 - - type: manhattan_pearson - value: 58.579416468759185 - - type: manhattan_spearman - value: 62.459738981727 - task: - type: STS - - dataset: - config: ar - name: MTEB STS22 (ar) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 48.76009839569795 - - type: cos_sim_spearman - value: 56.65188431953149 - - type: euclidean_pearson - value: 50.997682160915595 - - type: euclidean_spearman - value: 55.99910008818135 - - type: manhattan_pearson - value: 50.76220659606342 - - type: manhattan_spearman - value: 55.517347595391456 - task: - type: STS - - dataset: - config: ru - name: MTEB STS22 (ru) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cosine_pearson - value: 50.724322379215934 - - type: cosine_spearman - value: 59.90449732164651 - - type: euclidean_pearson - value: 50.227545226784024 - - type: euclidean_spearman - value: 59.898906527601085 - - type: main_score - value: 59.90449732164651 - - type: manhattan_pearson - value: 50.21762139819405 - - type: manhattan_spearman - value: 59.761039813759 - - type: pearson - value: 50.724322379215934 - - type: spearman - value: 59.90449732164651 - task: - type: STS - - dataset: - config: zh - name: MTEB STS22 (zh) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 54.717524559088005 - - type: cos_sim_spearman - value: 66.83570886252286 - - type: euclidean_pearson - value: 58.41338625505467 - - type: euclidean_spearman - value: 66.68991427704938 - - type: manhattan_pearson - value: 58.78638572916807 - - type: manhattan_spearman - value: 66.58684161046335 - task: - type: STS - - dataset: - config: fr - name: MTEB STS22 (fr) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 73.2962042954962 - - type: cos_sim_spearman - value: 76.58255504852025 - - type: euclidean_pearson - value: 75.70983192778257 - - type: euclidean_spearman - value: 77.4547684870542 - - type: manhattan_pearson - value: 75.75565853870485 - - type: manhattan_spearman - value: 76.90208974949428 - task: - type: STS - - dataset: - config: de-en - name: MTEB STS22 (de-en) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 54.47396266924846 - - type: cos_sim_spearman - value: 56.492267162048606 - - type: euclidean_pearson - value: 55.998505203070195 - - type: euclidean_spearman - value: 56.46447012960222 - - type: manhattan_pearson - value: 54.873172394430995 - - type: manhattan_spearman - value: 56.58111534551218 - task: - type: STS - - dataset: - config: es-en - name: MTEB STS22 (es-en) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 69.87177267688686 - - type: cos_sim_spearman - value: 74.57160943395763 - - type: euclidean_pearson - value: 70.88330406826788 - - type: euclidean_spearman - value: 74.29767636038422 - - type: manhattan_pearson - value: 71.38245248369536 - - type: manhattan_spearman - value: 74.53102232732175 - task: - type: STS - - dataset: - config: it - name: MTEB STS22 (it) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 72.80225656959544 - - type: cos_sim_spearman - value: 76.52646173725735 - - type: euclidean_pearson - value: 73.95710720200799 - - type: euclidean_spearman - value: 76.54040031984111 - - type: manhattan_pearson - value: 73.89679971946774 - - type: manhattan_spearman - value: 76.60886958161574 - task: - type: STS - - dataset: - config: pl-en - name: MTEB STS22 (pl-en) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 70.70844249898789 - - type: cos_sim_spearman - value: 72.68571783670241 - - type: euclidean_pearson - value: 72.38800772441031 - - type: euclidean_spearman - value: 72.86804422703312 - - type: manhattan_pearson - value: 71.29840508203515 - - type: manhattan_spearman - value: 71.86264441749513 - task: - type: STS - - dataset: - config: zh-en - name: MTEB STS22 (zh-en) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 58.647478923935694 - - type: cos_sim_spearman - value: 63.74453623540931 - - type: euclidean_pearson - value: 59.60138032437505 - - type: euclidean_spearman - value: 63.947930832166065 - - type: manhattan_pearson - value: 58.59735509491861 - - type: manhattan_spearman - value: 62.082503844627404 - task: - type: STS - - dataset: - config: es-it - name: MTEB STS22 (es-it) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 65.8722516867162 - - type: cos_sim_spearman - value: 71.81208592523012 - - type: euclidean_pearson - value: 67.95315252165956 - - type: euclidean_spearman - value: 73.00749822046009 - - type: manhattan_pearson - value: 68.07884688638924 - - type: manhattan_spearman - value: 72.34210325803069 - task: - type: STS - - dataset: - config: de-fr - name: MTEB STS22 (de-fr) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 54.5405814240949 - - type: cos_sim_spearman - value: 60.56838649023775 - - type: euclidean_pearson - value: 53.011731611314104 - - type: euclidean_spearman - value: 58.533194841668426 - - type: manhattan_pearson - value: 53.623067729338494 - - type: manhattan_spearman - value: 58.018756154446926 - task: - type: STS - - dataset: - config: de-pl - name: MTEB STS22 (de-pl) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 13.611046866216112 - - type: cos_sim_spearman - value: 28.238192909158492 - - type: euclidean_pearson - value: 22.16189199885129 - - type: euclidean_spearman - value: 35.012895679076564 - - type: manhattan_pearson - value: 21.969771178698387 - - type: manhattan_spearman - value: 32.456985088607475 - task: - type: STS - - dataset: - config: fr-pl - name: MTEB STS22 (fr-pl) - revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cos_sim_pearson - value: 74.58077407011655 - - type: cos_sim_spearman - value: 84.51542547285167 - - type: euclidean_pearson - value: 74.64613843596234 - - type: euclidean_spearman - value: 84.51542547285167 - - type: manhattan_pearson - value: 75.15335973101396 - - type: manhattan_spearman - value: 84.51542547285167 - task: - type: STS - - dataset: - config: default - name: MTEB STSBenchmark - revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831 - split: test - type: mteb/stsbenchmark-sts - metrics: - - type: cos_sim_pearson - value: 82.0739825531578 - - type: cos_sim_spearman - value: 84.01057479311115 - - type: euclidean_pearson - value: 83.85453227433344 - - type: euclidean_spearman - value: 84.01630226898655 - - type: manhattan_pearson - value: 83.75323603028978 - - type: manhattan_spearman - value: 83.89677983727685 - task: - type: STS - - dataset: - config: default - name: MTEB SciDocsRR - revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab - split: test - type: mteb/scidocs-reranking - metrics: - - type: map - value: 78.12945623123957 - - type: mrr - value: 93.87738713719106 - task: - type: Reranking - - dataset: - config: default - name: MTEB SciFact - revision: None - split: test - type: scifact - metrics: - - type: map_at_1 - value: 52.983000000000004 - - type: map_at_10 - value: 62.946000000000005 - - type: map_at_100 - value: 63.514 - - type: map_at_1000 - value: 63.554 - - type: map_at_3 - value: 60.183 - - type: map_at_5 - value: 61.672000000000004 - - type: mrr_at_1 - value: 55.667 - - type: mrr_at_10 - value: 64.522 - - type: mrr_at_100 - value: 64.957 - - type: mrr_at_1000 - value: 64.995 - - type: mrr_at_3 - value: 62.388999999999996 - - type: mrr_at_5 - value: 63.639 - - type: ndcg_at_1 - value: 55.667 - - type: ndcg_at_10 - value: 67.704 - - type: ndcg_at_100 - value: 70.299 - - type: ndcg_at_1000 - value: 71.241 - - type: ndcg_at_3 - value: 62.866 - - type: ndcg_at_5 - value: 65.16999999999999 - - type: precision_at_1 - value: 55.667 - - type: precision_at_10 - value: 9.033 - - type: precision_at_100 - value: 1.053 - - type: precision_at_1000 - value: 0.11299999999999999 - - type: precision_at_3 - value: 24.444 - - type: precision_at_5 - value: 16.133 - - type: recall_at_1 - value: 52.983000000000004 - - type: recall_at_10 - value: 80.656 - - type: recall_at_100 - value: 92.5 - - type: recall_at_1000 - value: 99.667 - - type: recall_at_3 - value: 67.744 - - type: recall_at_5 - value: 73.433 - task: - type: Retrieval - - dataset: - config: default - name: MTEB SprintDuplicateQuestions - revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 - split: test - type: mteb/sprintduplicatequestions-pairclassification - metrics: - - type: cos_sim_accuracy - value: 99.72772277227723 - - type: cos_sim_ap - value: 92.17845897992215 - - type: cos_sim_f1 - value: 85.9746835443038 - - type: cos_sim_precision - value: 87.07692307692308 - - type: cos_sim_recall - value: 84.89999999999999 - - type: dot_accuracy - value: 99.3039603960396 - - type: dot_ap - value: 60.70244020124878 - - type: dot_f1 - value: 59.92742353551063 - - type: dot_precision - value: 62.21743810548978 - - type: dot_recall - value: 57.8 - - type: euclidean_accuracy - value: 99.71683168316832 - - type: euclidean_ap - value: 91.53997039964659 - - type: euclidean_f1 - value: 84.88372093023257 - - type: euclidean_precision - value: 90.02242152466367 - - type: euclidean_recall - value: 80.30000000000001 - - type: manhattan_accuracy - value: 99.72376237623763 - - type: manhattan_ap - value: 91.80756777790289 - - type: manhattan_f1 - value: 85.48468106479157 - - type: manhattan_precision - value: 85.8728557013118 - - type: manhattan_recall - value: 85.1 - - type: max_accuracy - value: 99.72772277227723 - - type: max_ap - value: 92.17845897992215 - - type: max_f1 - value: 85.9746835443038 - task: - type: PairClassification - - dataset: - config: default - name: MTEB StackExchangeClustering - revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 - split: test - type: mteb/stackexchange-clustering - metrics: - - type: v_measure - value: 53.52464042600003 - task: - type: Clustering - - dataset: - config: default - name: MTEB StackExchangeClusteringP2P - revision: 815ca46b2622cec33ccafc3735d572c266efdb44 - split: test - type: mteb/stackexchange-clustering-p2p - metrics: - - type: v_measure - value: 32.071631948736 - task: - type: Clustering - - dataset: - config: default - name: MTEB StackOverflowDupQuestions - revision: e185fbe320c72810689fc5848eb6114e1ef5ec69 - split: test - type: mteb/stackoverflowdupquestions-reranking - metrics: - - type: map - value: 49.19552407604654 - - type: mrr - value: 49.95269130379425 - task: - type: Reranking - - dataset: - config: default - name: MTEB SummEval - revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c - split: test - type: mteb/summeval - metrics: - - type: cos_sim_pearson - value: 29.345293033095427 - - type: cos_sim_spearman - value: 29.976931423258403 - - type: dot_pearson - value: 27.047078008958408 - - type: dot_spearman - value: 27.75894368380218 - task: - type: Summarization - - dataset: - config: default - name: MTEB TRECCOVID - revision: None - split: test - type: trec-covid - metrics: - - type: map_at_1 - value: 0.22 - - type: map_at_10 - value: 1.706 - - type: map_at_100 - value: 9.634 - - type: map_at_1000 - value: 23.665 - - type: map_at_3 - value: 0.5950000000000001 - - type: map_at_5 - value: 0.95 - - type: mrr_at_1 - value: 86.0 - - type: mrr_at_10 - value: 91.8 - - type: mrr_at_100 - value: 91.8 - - type: mrr_at_1000 - value: 91.8 - - type: mrr_at_3 - value: 91.0 - - type: mrr_at_5 - value: 91.8 - - type: ndcg_at_1 - value: 80.0 - - type: ndcg_at_10 - value: 72.573 - - type: ndcg_at_100 - value: 53.954 - - type: ndcg_at_1000 - value: 47.760999999999996 - - type: ndcg_at_3 - value: 76.173 - - type: ndcg_at_5 - value: 75.264 - - type: precision_at_1 - value: 86.0 - - type: precision_at_10 - value: 76.4 - - type: precision_at_100 - value: 55.50000000000001 - - type: precision_at_1000 - value: 21.802 - - type: precision_at_3 - value: 81.333 - - type: precision_at_5 - value: 80.4 - - type: recall_at_1 - value: 0.22 - - type: recall_at_10 - value: 1.925 - - type: recall_at_100 - value: 12.762 - - type: recall_at_1000 - value: 44.946000000000005 - - type: recall_at_3 - value: 0.634 - - type: recall_at_5 - value: 1.051 - task: - type: Retrieval - - dataset: - config: sqi-eng - name: MTEB Tatoeba (sqi-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 91.0 - - type: f1 - value: 88.55666666666666 - - type: precision - value: 87.46166666666667 - - type: recall - value: 91.0 - task: - type: BitextMining - - dataset: - config: fry-eng - name: MTEB Tatoeba (fry-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 57.22543352601156 - - type: f1 - value: 51.03220478943021 - - type: precision - value: 48.8150289017341 - - type: recall - value: 57.22543352601156 - task: - type: BitextMining - - dataset: - config: kur-eng - name: MTEB Tatoeba (kur-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 46.58536585365854 - - type: f1 - value: 39.66870798578116 - - type: precision - value: 37.416085946573745 - - type: recall - value: 46.58536585365854 - task: - type: BitextMining - - dataset: - config: tur-eng - name: MTEB Tatoeba (tur-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 89.7 - - type: f1 - value: 86.77999999999999 - - type: precision - value: 85.45333333333332 - - type: recall - value: 89.7 - task: - type: BitextMining - - dataset: - config: deu-eng - name: MTEB Tatoeba (deu-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 97.39999999999999 - - type: f1 - value: 96.58333333333331 - - type: precision - value: 96.2 - - type: recall - value: 97.39999999999999 - task: - type: BitextMining - - dataset: - config: nld-eng - name: MTEB Tatoeba (nld-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 92.4 - - type: f1 - value: 90.3 - - type: precision - value: 89.31666666666668 - - type: recall - value: 92.4 - task: - type: BitextMining - - dataset: - config: ron-eng - name: MTEB Tatoeba (ron-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 86.9 - - type: f1 - value: 83.67190476190476 - - type: precision - value: 82.23333333333332 - - type: recall - value: 86.9 - task: - type: BitextMining - - dataset: - config: ang-eng - name: MTEB Tatoeba (ang-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 50.0 - - type: f1 - value: 42.23229092632078 - - type: precision - value: 39.851634683724235 - - type: recall - value: 50.0 - task: - type: BitextMining - - dataset: - config: ido-eng - name: MTEB Tatoeba (ido-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 76.3 - - type: f1 - value: 70.86190476190477 - - type: precision - value: 68.68777777777777 - - type: recall - value: 76.3 - task: - type: BitextMining - - dataset: - config: jav-eng - name: MTEB Tatoeba (jav-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 57.073170731707314 - - type: f1 - value: 50.658958927251604 - - type: precision - value: 48.26480836236933 - - type: recall - value: 57.073170731707314 - task: - type: BitextMining - - dataset: - config: isl-eng - name: MTEB Tatoeba (isl-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 68.2 - - type: f1 - value: 62.156507936507936 - - type: precision - value: 59.84964285714286 - - type: recall - value: 68.2 - task: - type: BitextMining - - dataset: - config: slv-eng - name: MTEB Tatoeba (slv-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 77.52126366950182 - - type: f1 - value: 72.8496210148701 - - type: precision - value: 70.92171498003819 - - type: recall - value: 77.52126366950182 - task: - type: BitextMining - - dataset: - config: cym-eng - name: MTEB Tatoeba (cym-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 70.78260869565217 - - type: f1 - value: 65.32422360248447 - - type: precision - value: 63.063067367415194 - - type: recall - value: 70.78260869565217 - task: - type: BitextMining - - dataset: - config: kaz-eng - name: MTEB Tatoeba (kaz-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 78.43478260869566 - - type: f1 - value: 73.02608695652172 - - type: precision - value: 70.63768115942028 - - type: recall - value: 78.43478260869566 - task: - type: BitextMining - - dataset: - config: est-eng - name: MTEB Tatoeba (est-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 60.9 - - type: f1 - value: 55.309753694581275 - - type: precision - value: 53.130476190476195 - - type: recall - value: 60.9 - task: - type: BitextMining - - dataset: - config: heb-eng - name: MTEB Tatoeba (heb-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 72.89999999999999 - - type: f1 - value: 67.92023809523809 - - type: precision - value: 65.82595238095237 - - type: recall - value: 72.89999999999999 - task: - type: BitextMining - - dataset: - config: gla-eng - name: MTEB Tatoeba (gla-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 46.80337756332931 - - type: f1 - value: 39.42174900558496 - - type: precision - value: 36.97101116280851 - - type: recall - value: 46.80337756332931 - task: - type: BitextMining - - dataset: - config: mar-eng - name: MTEB Tatoeba (mar-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 89.8 - - type: f1 - value: 86.79 - - type: precision - value: 85.375 - - type: recall - value: 89.8 - task: - type: BitextMining - - dataset: - config: lat-eng - name: MTEB Tatoeba (lat-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 47.199999999999996 - - type: f1 - value: 39.95484348984349 - - type: precision - value: 37.561071428571424 - - type: recall - value: 47.199999999999996 - task: - type: BitextMining - - dataset: - config: bel-eng - name: MTEB Tatoeba (bel-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 87.8 - - type: f1 - value: 84.68190476190475 - - type: precision - value: 83.275 - - type: recall - value: 87.8 - task: - type: BitextMining - - dataset: - config: pms-eng - name: MTEB Tatoeba (pms-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 48.76190476190476 - - type: f1 - value: 42.14965986394558 - - type: precision - value: 39.96743626743626 - - type: recall - value: 48.76190476190476 - task: - type: BitextMining - - dataset: - config: gle-eng - name: MTEB Tatoeba (gle-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 66.10000000000001 - - type: f1 - value: 59.58580086580086 - - type: precision - value: 57.150238095238095 - - type: recall - value: 66.10000000000001 - task: - type: BitextMining - - dataset: - config: pes-eng - name: MTEB Tatoeba (pes-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 87.3 - - type: f1 - value: 84.0 - - type: precision - value: 82.48666666666666 - - type: recall - value: 87.3 - task: - type: BitextMining - - dataset: - config: nob-eng - name: MTEB Tatoeba (nob-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 90.4 - - type: f1 - value: 87.79523809523809 - - type: precision - value: 86.6 - - type: recall - value: 90.4 - task: - type: BitextMining - - dataset: - config: bul-eng - name: MTEB Tatoeba (bul-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 87.0 - - type: f1 - value: 83.81 - - type: precision - value: 82.36666666666666 - - type: recall - value: 87.0 - task: - type: BitextMining - - dataset: - config: cbk-eng - name: MTEB Tatoeba (cbk-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 63.9 - - type: f1 - value: 57.76533189033189 - - type: precision - value: 55.50595238095239 - - type: recall - value: 63.9 - task: - type: BitextMining - - dataset: - config: hun-eng - name: MTEB Tatoeba (hun-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 76.1 - - type: f1 - value: 71.83690476190478 - - type: precision - value: 70.04928571428573 - - type: recall - value: 76.1 - task: - type: BitextMining - - dataset: - config: uig-eng - name: MTEB Tatoeba (uig-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 66.3 - - type: f1 - value: 59.32626984126984 - - type: precision - value: 56.62535714285713 - - type: recall - value: 66.3 - task: - type: BitextMining - - dataset: - config: rus-eng - name: MTEB Tatoeba (rus-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 92.10000000000001 - - type: f1 - value: 89.76666666666667 - - type: main_score - value: 89.76666666666667 - - type: precision - value: 88.64999999999999 - - type: recall - value: 92.10000000000001 - task: - type: BitextMining - - dataset: - config: spa-eng - name: MTEB Tatoeba (spa-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 93.10000000000001 - - type: f1 - value: 91.10000000000001 - - type: precision - value: 90.16666666666666 - - type: recall - value: 93.10000000000001 - task: - type: BitextMining - - dataset: - config: hye-eng - name: MTEB Tatoeba (hye-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 85.71428571428571 - - type: f1 - value: 82.29142600436403 - - type: precision - value: 80.8076626877166 - - type: recall - value: 85.71428571428571 - task: - type: BitextMining - - dataset: - config: tel-eng - name: MTEB Tatoeba (tel-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 88.88888888888889 - - type: f1 - value: 85.7834757834758 - - type: precision - value: 84.43732193732193 - - type: recall - value: 88.88888888888889 - task: - type: BitextMining - - dataset: - config: afr-eng - name: MTEB Tatoeba (afr-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 88.5 - - type: f1 - value: 85.67190476190476 - - type: precision - value: 84.43333333333332 - - type: recall - value: 88.5 - task: - type: BitextMining - - dataset: - config: mon-eng - name: MTEB Tatoeba (mon-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 82.72727272727273 - - type: f1 - value: 78.21969696969695 - - type: precision - value: 76.18181818181819 - - type: recall - value: 82.72727272727273 - task: - type: BitextMining - - dataset: - config: arz-eng - name: MTEB Tatoeba (arz-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 61.0062893081761 - - type: f1 - value: 55.13976240391334 - - type: precision - value: 52.92112499659669 - - type: recall - value: 61.0062893081761 - task: - type: BitextMining - - dataset: - config: hrv-eng - name: MTEB Tatoeba (hrv-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 89.5 - - type: f1 - value: 86.86666666666666 - - type: precision - value: 85.69166666666668 - - type: recall - value: 89.5 - task: - type: BitextMining - - dataset: - config: nov-eng - name: MTEB Tatoeba (nov-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 73.54085603112841 - - type: f1 - value: 68.56031128404669 - - type: precision - value: 66.53047989623866 - - type: recall - value: 73.54085603112841 - task: - type: BitextMining - - dataset: - config: gsw-eng - name: MTEB Tatoeba (gsw-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 43.58974358974359 - - type: f1 - value: 36.45299145299145 - - type: precision - value: 33.81155881155882 - - type: recall - value: 43.58974358974359 - task: - type: BitextMining - - dataset: - config: nds-eng - name: MTEB Tatoeba (nds-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 59.599999999999994 - - type: f1 - value: 53.264689754689755 - - type: precision - value: 50.869166666666665 - - type: recall - value: 59.599999999999994 - task: - type: BitextMining - - dataset: - config: ukr-eng - name: MTEB Tatoeba (ukr-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 85.2 - - type: f1 - value: 81.61666666666665 - - type: precision - value: 80.02833333333335 - - type: recall - value: 85.2 - task: - type: BitextMining - - dataset: - config: uzb-eng - name: MTEB Tatoeba (uzb-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 63.78504672897196 - - type: f1 - value: 58.00029669188548 - - type: precision - value: 55.815809968847354 - - type: recall - value: 63.78504672897196 - task: - type: BitextMining - - dataset: - config: lit-eng - name: MTEB Tatoeba (lit-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 66.5 - - type: f1 - value: 61.518333333333345 - - type: precision - value: 59.622363699102834 - - type: recall - value: 66.5 - task: - type: BitextMining - - dataset: - config: ina-eng - name: MTEB Tatoeba (ina-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 88.6 - - type: f1 - value: 85.60222222222221 - - type: precision - value: 84.27916666666665 - - type: recall - value: 88.6 - task: - type: BitextMining - - dataset: - config: lfn-eng - name: MTEB Tatoeba (lfn-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 58.699999999999996 - - type: f1 - value: 52.732375957375965 - - type: precision - value: 50.63214035964035 - - type: recall - value: 58.699999999999996 - task: - type: BitextMining - - dataset: - config: zsm-eng - name: MTEB Tatoeba (zsm-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 92.10000000000001 - - type: f1 - value: 89.99666666666667 - - type: precision - value: 89.03333333333333 - - type: recall - value: 92.10000000000001 - task: - type: BitextMining - - dataset: - config: ita-eng - name: MTEB Tatoeba (ita-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 90.10000000000001 - - type: f1 - value: 87.55666666666667 - - type: precision - value: 86.36166666666668 - - type: recall - value: 90.10000000000001 - task: - type: BitextMining - - dataset: - config: cmn-eng - name: MTEB Tatoeba (cmn-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 91.4 - - type: f1 - value: 88.89000000000001 - - type: precision - value: 87.71166666666666 - - type: recall - value: 91.4 - task: - type: BitextMining - - dataset: - config: lvs-eng - name: MTEB Tatoeba (lvs-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 65.7 - - type: f1 - value: 60.67427750410509 - - type: precision - value: 58.71785714285714 - - type: recall - value: 65.7 - task: - type: BitextMining - - dataset: - config: glg-eng - name: MTEB Tatoeba (glg-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 85.39999999999999 - - type: f1 - value: 81.93190476190475 - - type: precision - value: 80.37833333333333 - - type: recall - value: 85.39999999999999 - task: - type: BitextMining - - dataset: - config: ceb-eng - name: MTEB Tatoeba (ceb-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 47.833333333333336 - - type: f1 - value: 42.006625781625786 - - type: precision - value: 40.077380952380956 - - type: recall - value: 47.833333333333336 - task: - type: BitextMining - - dataset: - config: bre-eng - name: MTEB Tatoeba (bre-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 10.4 - - type: f1 - value: 8.24465007215007 - - type: precision - value: 7.664597069597071 - - type: recall - value: 10.4 - task: - type: BitextMining - - dataset: - config: ben-eng - name: MTEB Tatoeba (ben-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 82.6 - - type: f1 - value: 77.76333333333334 - - type: precision - value: 75.57833333333332 - - type: recall - value: 82.6 - task: - type: BitextMining - - dataset: - config: swg-eng - name: MTEB Tatoeba (swg-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 52.67857142857143 - - type: f1 - value: 44.302721088435376 - - type: precision - value: 41.49801587301587 - - type: recall - value: 52.67857142857143 - task: - type: BitextMining - - dataset: - config: arq-eng - name: MTEB Tatoeba (arq-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 28.3205268935236 - - type: f1 - value: 22.426666605171157 - - type: precision - value: 20.685900116470915 - - type: recall - value: 28.3205268935236 - task: - type: BitextMining - - dataset: - config: kab-eng - name: MTEB Tatoeba (kab-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 22.7 - - type: f1 - value: 17.833970473970474 - - type: precision - value: 16.407335164835164 - - type: recall - value: 22.7 - task: - type: BitextMining - - dataset: - config: fra-eng - name: MTEB Tatoeba (fra-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 92.2 - - type: f1 - value: 89.92999999999999 - - type: precision - value: 88.87 - - type: recall - value: 92.2 - task: - type: BitextMining - - dataset: - config: por-eng - name: MTEB Tatoeba (por-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 91.4 - - type: f1 - value: 89.25 - - type: precision - value: 88.21666666666667 - - type: recall - value: 91.4 - task: - type: BitextMining - - dataset: - config: tat-eng - name: MTEB Tatoeba (tat-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 69.19999999999999 - - type: f1 - value: 63.38269841269841 - - type: precision - value: 61.14773809523809 - - type: recall - value: 69.19999999999999 - task: - type: BitextMining - - dataset: - config: oci-eng - name: MTEB Tatoeba (oci-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 48.8 - - type: f1 - value: 42.839915639915645 - - type: precision - value: 40.770287114845935 - - type: recall - value: 48.8 - task: - type: BitextMining - - dataset: - config: pol-eng - name: MTEB Tatoeba (pol-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 88.8 - - type: f1 - value: 85.90666666666668 - - type: precision - value: 84.54166666666666 - - type: recall - value: 88.8 - task: - type: BitextMining - - dataset: - config: war-eng - name: MTEB Tatoeba (war-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 46.6 - - type: f1 - value: 40.85892920804686 - - type: precision - value: 38.838223114604695 - - type: recall - value: 46.6 - task: - type: BitextMining - - dataset: - config: aze-eng - name: MTEB Tatoeba (aze-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 84.0 - - type: f1 - value: 80.14190476190475 - - type: precision - value: 78.45333333333333 - - type: recall - value: 84.0 - task: - type: BitextMining - - dataset: - config: vie-eng - name: MTEB Tatoeba (vie-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 90.5 - - type: f1 - value: 87.78333333333333 - - type: precision - value: 86.5 - - type: recall - value: 90.5 - task: - type: BitextMining - - dataset: - config: nno-eng - name: MTEB Tatoeba (nno-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 74.5 - - type: f1 - value: 69.48397546897547 - - type: precision - value: 67.51869047619049 - - type: recall - value: 74.5 - task: - type: BitextMining - - dataset: - config: cha-eng - name: MTEB Tatoeba (cha-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 32.846715328467155 - - type: f1 - value: 27.828177499710343 - - type: precision - value: 26.63451511991658 - - type: recall - value: 32.846715328467155 - task: - type: BitextMining - - dataset: - config: mhr-eng - name: MTEB Tatoeba (mhr-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 8.0 - - type: f1 - value: 6.07664116764988 - - type: precision - value: 5.544177607179943 - - type: recall - value: 8.0 - task: - type: BitextMining - - dataset: - config: dan-eng - name: MTEB Tatoeba (dan-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 87.6 - - type: f1 - value: 84.38555555555554 - - type: precision - value: 82.91583333333334 - - type: recall - value: 87.6 - task: - type: BitextMining - - dataset: - config: ell-eng - name: MTEB Tatoeba (ell-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 87.5 - - type: f1 - value: 84.08333333333331 - - type: precision - value: 82.47333333333333 - - type: recall - value: 87.5 - task: - type: BitextMining - - dataset: - config: amh-eng - name: MTEB Tatoeba (amh-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 80.95238095238095 - - type: f1 - value: 76.13095238095238 - - type: precision - value: 74.05753968253967 - - type: recall - value: 80.95238095238095 - task: - type: BitextMining - - dataset: - config: pam-eng - name: MTEB Tatoeba (pam-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 8.799999999999999 - - type: f1 - value: 6.971422975172975 - - type: precision - value: 6.557814916172301 - - type: recall - value: 8.799999999999999 - task: - type: BitextMining - - dataset: - config: hsb-eng - name: MTEB Tatoeba (hsb-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 44.099378881987576 - - type: f1 - value: 37.01649742022413 - - type: precision - value: 34.69420618488942 - - type: recall - value: 44.099378881987576 - task: - type: BitextMining - - dataset: - config: srp-eng - name: MTEB Tatoeba (srp-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 84.3 - - type: f1 - value: 80.32666666666667 - - type: precision - value: 78.60666666666665 - - type: recall - value: 84.3 - task: - type: BitextMining - - dataset: - config: epo-eng - name: MTEB Tatoeba (epo-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 92.5 - - type: f1 - value: 90.49666666666666 - - type: precision - value: 89.56666666666668 - - type: recall - value: 92.5 - task: - type: BitextMining - - dataset: - config: kzj-eng - name: MTEB Tatoeba (kzj-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 10.0 - - type: f1 - value: 8.268423529875141 - - type: precision - value: 7.878118605532398 - - type: recall - value: 10.0 - task: - type: BitextMining - - dataset: - config: awa-eng - name: MTEB Tatoeba (awa-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 79.22077922077922 - - type: f1 - value: 74.27128427128426 - - type: precision - value: 72.28715728715729 - - type: recall - value: 79.22077922077922 - task: - type: BitextMining - - dataset: - config: fao-eng - name: MTEB Tatoeba (fao-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 65.64885496183206 - - type: f1 - value: 58.87495456197747 - - type: precision - value: 55.992366412213734 - - type: recall - value: 65.64885496183206 - task: - type: BitextMining - - dataset: - config: mal-eng - name: MTEB Tatoeba (mal-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 96.06986899563319 - - type: f1 - value: 94.78408539543909 - - type: precision - value: 94.15332362930616 - - type: recall - value: 96.06986899563319 - task: - type: BitextMining - - dataset: - config: ile-eng - name: MTEB Tatoeba (ile-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 77.2 - - type: f1 - value: 71.72571428571428 - - type: precision - value: 69.41000000000001 - - type: recall - value: 77.2 - task: - type: BitextMining - - dataset: - config: bos-eng - name: MTEB Tatoeba (bos-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 86.4406779661017 - - type: f1 - value: 83.2391713747646 - - type: precision - value: 81.74199623352166 - - type: recall - value: 86.4406779661017 - task: - type: BitextMining - - dataset: - config: cor-eng - name: MTEB Tatoeba (cor-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 8.4 - - type: f1 - value: 6.017828743398003 - - type: precision - value: 5.4829865484756795 - - type: recall - value: 8.4 - task: - type: BitextMining - - dataset: - config: cat-eng - name: MTEB Tatoeba (cat-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 83.5 - - type: f1 - value: 79.74833333333333 - - type: precision - value: 78.04837662337664 - - type: recall - value: 83.5 - task: - type: BitextMining - - dataset: - config: eus-eng - name: MTEB Tatoeba (eus-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 60.4 - - type: f1 - value: 54.467301587301584 - - type: precision - value: 52.23242424242424 - - type: recall - value: 60.4 - task: - type: BitextMining - - dataset: - config: yue-eng - name: MTEB Tatoeba (yue-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 74.9 - - type: f1 - value: 69.68699134199134 - - type: precision - value: 67.59873015873016 - - type: recall - value: 74.9 - task: - type: BitextMining - - dataset: - config: swe-eng - name: MTEB Tatoeba (swe-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 88.0 - - type: f1 - value: 84.9652380952381 - - type: precision - value: 83.66166666666666 - - type: recall - value: 88.0 - task: - type: BitextMining - - dataset: - config: dtp-eng - name: MTEB Tatoeba (dtp-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 9.1 - - type: f1 - value: 7.681244588744588 - - type: precision - value: 7.370043290043291 - - type: recall - value: 9.1 - task: - type: BitextMining - - dataset: - config: kat-eng - name: MTEB Tatoeba (kat-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 80.9651474530831 - - type: f1 - value: 76.84220605132133 - - type: precision - value: 75.19606398962966 - - type: recall - value: 80.9651474530831 - task: - type: BitextMining - - dataset: - config: jpn-eng - name: MTEB Tatoeba (jpn-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 86.9 - - type: f1 - value: 83.705 - - type: precision - value: 82.3120634920635 - - type: recall - value: 86.9 - task: - type: BitextMining - - dataset: - config: csb-eng - name: MTEB Tatoeba (csb-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 29.64426877470356 - - type: f1 - value: 23.98763072676116 - - type: precision - value: 22.506399397703746 - - type: recall - value: 29.64426877470356 - task: - type: BitextMining - - dataset: - config: xho-eng - name: MTEB Tatoeba (xho-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 70.4225352112676 - - type: f1 - value: 62.84037558685445 - - type: precision - value: 59.56572769953053 - - type: recall - value: 70.4225352112676 - task: - type: BitextMining - - dataset: - config: orv-eng - name: MTEB Tatoeba (orv-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 19.64071856287425 - - type: f1 - value: 15.125271011207756 - - type: precision - value: 13.865019261197494 - - type: recall - value: 19.64071856287425 - task: - type: BitextMining - - dataset: - config: ind-eng - name: MTEB Tatoeba (ind-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 90.2 - - type: f1 - value: 87.80666666666666 - - type: precision - value: 86.70833333333331 - - type: recall - value: 90.2 - task: - type: BitextMining - - dataset: - config: tuk-eng - name: MTEB Tatoeba (tuk-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 23.15270935960591 - - type: f1 - value: 18.407224958949097 - - type: precision - value: 16.982385430661292 - - type: recall - value: 23.15270935960591 - task: - type: BitextMining - - dataset: - config: max-eng - name: MTEB Tatoeba (max-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 55.98591549295775 - - type: f1 - value: 49.94718309859154 - - type: precision - value: 47.77864154624717 - - type: recall - value: 55.98591549295775 - task: - type: BitextMining - - dataset: - config: swh-eng - name: MTEB Tatoeba (swh-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 73.07692307692307 - - type: f1 - value: 66.74358974358974 - - type: precision - value: 64.06837606837607 - - type: recall - value: 73.07692307692307 - task: - type: BitextMining - - dataset: - config: hin-eng - name: MTEB Tatoeba (hin-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 94.89999999999999 - - type: f1 - value: 93.25 - - type: precision - value: 92.43333333333332 - - type: recall - value: 94.89999999999999 - task: - type: BitextMining - - dataset: - config: dsb-eng - name: MTEB Tatoeba (dsb-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 37.78705636743215 - - type: f1 - value: 31.63899658680452 - - type: precision - value: 29.72264397629742 - - type: recall - value: 37.78705636743215 - task: - type: BitextMining - - dataset: - config: ber-eng - name: MTEB Tatoeba (ber-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 21.6 - - type: f1 - value: 16.91697302697303 - - type: precision - value: 15.71225147075147 - - type: recall - value: 21.6 - task: - type: BitextMining - - dataset: - config: tam-eng - name: MTEB Tatoeba (tam-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 85.01628664495115 - - type: f1 - value: 81.38514037536838 - - type: precision - value: 79.83170466883823 - - type: recall - value: 85.01628664495115 - task: - type: BitextMining - - dataset: - config: slk-eng - name: MTEB Tatoeba (slk-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 83.39999999999999 - - type: f1 - value: 79.96380952380952 - - type: precision - value: 78.48333333333333 - - type: recall - value: 83.39999999999999 - task: - type: BitextMining - - dataset: - config: tgl-eng - name: MTEB Tatoeba (tgl-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 83.2 - - type: f1 - value: 79.26190476190476 - - type: precision - value: 77.58833333333334 - - type: recall - value: 83.2 - task: - type: BitextMining - - dataset: - config: ast-eng - name: MTEB Tatoeba (ast-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 75.59055118110236 - - type: f1 - value: 71.66854143232096 - - type: precision - value: 70.30183727034121 - - type: recall - value: 75.59055118110236 - task: - type: BitextMining - - dataset: - config: mkd-eng - name: MTEB Tatoeba (mkd-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 65.5 - - type: f1 - value: 59.26095238095238 - - type: precision - value: 56.81909090909092 - - type: recall - value: 65.5 - task: - type: BitextMining - - dataset: - config: khm-eng - name: MTEB Tatoeba (khm-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 55.26315789473685 - - type: f1 - value: 47.986523325858506 - - type: precision - value: 45.33950006595436 - - type: recall - value: 55.26315789473685 - task: - type: BitextMining - - dataset: - config: ces-eng - name: MTEB Tatoeba (ces-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 82.89999999999999 - - type: f1 - value: 78.835 - - type: precision - value: 77.04761904761905 - - type: recall - value: 82.89999999999999 - task: - type: BitextMining - - dataset: - config: tzl-eng - name: MTEB Tatoeba (tzl-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 43.269230769230774 - - type: f1 - value: 36.20421245421245 - - type: precision - value: 33.57371794871795 - - type: recall - value: 43.269230769230774 - task: - type: BitextMining - - dataset: - config: urd-eng - name: MTEB Tatoeba (urd-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 88.0 - - type: f1 - value: 84.70666666666666 - - type: precision - value: 83.23166666666665 - - type: recall - value: 88.0 - task: - type: BitextMining - - dataset: - config: ara-eng - name: MTEB Tatoeba (ara-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 77.4 - - type: f1 - value: 72.54666666666667 - - type: precision - value: 70.54318181818181 - - type: recall - value: 77.4 - task: - type: BitextMining - - dataset: - config: kor-eng - name: MTEB Tatoeba (kor-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 78.60000000000001 - - type: f1 - value: 74.1588888888889 - - type: precision - value: 72.30250000000001 - - type: recall - value: 78.60000000000001 - task: - type: BitextMining - - dataset: - config: yid-eng - name: MTEB Tatoeba (yid-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 72.40566037735849 - - type: f1 - value: 66.82587328813744 - - type: precision - value: 64.75039308176099 - - type: recall - value: 72.40566037735849 - task: - type: BitextMining - - dataset: - config: fin-eng - name: MTEB Tatoeba (fin-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 73.8 - - type: f1 - value: 68.56357142857144 - - type: precision - value: 66.3178822055138 - - type: recall - value: 73.8 - task: - type: BitextMining - - dataset: - config: tha-eng - name: MTEB Tatoeba (tha-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 91.78832116788321 - - type: f1 - value: 89.3552311435523 - - type: precision - value: 88.20559610705597 - - type: recall - value: 91.78832116788321 - task: - type: BitextMining - - dataset: - config: wuu-eng - name: MTEB Tatoeba (wuu-eng) - revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553 - split: test - type: mteb/tatoeba-bitext-mining - metrics: - - type: accuracy - value: 74.3 - - type: f1 - value: 69.05085581085581 - - type: precision - value: 66.955 - - type: recall - value: 74.3 - task: - type: BitextMining - - dataset: - config: default - name: MTEB Touche2020 - revision: None - split: test - type: webis-touche2020 - metrics: - - type: map_at_1 - value: 2.896 - - type: map_at_10 - value: 8.993 - - type: map_at_100 - value: 14.133999999999999 - - type: map_at_1000 - value: 15.668000000000001 - - type: map_at_3 - value: 5.862 - - type: map_at_5 - value: 7.17 - - type: mrr_at_1 - value: 34.694 - - type: mrr_at_10 - value: 42.931000000000004 - - type: mrr_at_100 - value: 44.81 - - type: mrr_at_1000 - value: 44.81 - - type: mrr_at_3 - value: 38.435 - - type: mrr_at_5 - value: 41.701 - - type: ndcg_at_1 - value: 31.633 - - type: ndcg_at_10 - value: 21.163 - - type: ndcg_at_100 - value: 33.306000000000004 - - type: ndcg_at_1000 - value: 45.275999999999996 - - type: ndcg_at_3 - value: 25.685999999999996 - - type: ndcg_at_5 - value: 23.732 - - type: precision_at_1 - value: 34.694 - - type: precision_at_10 - value: 17.755000000000003 - - type: precision_at_100 - value: 6.938999999999999 - - type: precision_at_1000 - value: 1.48 - - type: precision_at_3 - value: 25.85 - - type: precision_at_5 - value: 23.265 - - type: recall_at_1 - value: 2.896 - - type: recall_at_10 - value: 13.333999999999998 - - type: recall_at_100 - value: 43.517 - - type: recall_at_1000 - value: 79.836 - - type: recall_at_3 - value: 6.306000000000001 - - type: recall_at_5 - value: 8.825 - task: - type: Retrieval - - dataset: - config: default - name: MTEB ToxicConversationsClassification - revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c - split: test - type: mteb/toxic_conversations_50k - metrics: - - type: accuracy - value: 69.3874 - - type: ap - value: 13.829909072469423 - - type: f1 - value: 53.54534203543492 - task: - type: Classification - - dataset: - config: default - name: MTEB TweetSentimentExtractionClassification - revision: d604517c81ca91fe16a244d1248fc021f9ecee7a - split: test - type: mteb/tweet_sentiment_extraction - metrics: - - type: accuracy - value: 62.62026032823995 - - type: f1 - value: 62.85251350485221 - task: - type: Classification - - dataset: - config: default - name: MTEB TwentyNewsgroupsClustering - revision: 6125ec4e24fa026cec8a478383ee943acfbd5449 - split: test - type: mteb/twentynewsgroups-clustering - metrics: - - type: v_measure - value: 33.21527881409797 - task: - type: Clustering - - dataset: - config: default - name: MTEB TwitterSemEval2015 - revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1 - split: test - type: mteb/twittersemeval2015-pairclassification - metrics: - - type: cos_sim_accuracy - value: 84.97943613280086 - - type: cos_sim_ap - value: 70.75454316885921 - - type: cos_sim_f1 - value: 65.38274012676743 - - type: cos_sim_precision - value: 60.761214318078835 - - type: cos_sim_recall - value: 70.76517150395777 - - type: dot_accuracy - value: 79.0546581629612 - - type: dot_ap - value: 47.3197121792147 - - type: dot_f1 - value: 49.20106524633821 - - type: dot_precision - value: 42.45499808502489 - - type: dot_recall - value: 58.49604221635884 - - type: euclidean_accuracy - value: 85.08076533349228 - - type: euclidean_ap - value: 70.95016106374474 - - type: euclidean_f1 - value: 65.43987900176455 - - type: euclidean_precision - value: 62.64478764478765 - - type: euclidean_recall - value: 68.49604221635884 - - type: manhattan_accuracy - value: 84.93771234428085 - - type: manhattan_ap - value: 70.63668388755362 - - type: manhattan_f1 - value: 65.23895401262398 - - type: manhattan_precision - value: 56.946084218811485 - - type: manhattan_recall - value: 76.35883905013192 - - type: max_accuracy - value: 85.08076533349228 - - type: max_ap - value: 70.95016106374474 - - type: max_f1 - value: 65.43987900176455 - task: - type: PairClassification - - dataset: - config: default - name: MTEB TwitterURLCorpus - revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf - split: test - type: mteb/twitterurlcorpus-pairclassification - metrics: - - type: cos_sim_accuracy - value: 88.69096130709822 - - type: cos_sim_ap - value: 84.82526278228542 - - type: cos_sim_f1 - value: 77.65485060585536 - - type: cos_sim_precision - value: 75.94582658619167 - - type: cos_sim_recall - value: 79.44256236526024 - - type: dot_accuracy - value: 80.97954748321496 - - type: dot_ap - value: 64.81642914145866 - - type: dot_f1 - value: 60.631996987229975 - - type: dot_precision - value: 54.5897293631712 - - type: dot_recall - value: 68.17831844779796 - - type: euclidean_accuracy - value: 88.6987231730508 - - type: euclidean_ap - value: 84.80003825477253 - - type: euclidean_f1 - value: 77.67194179854496 - - type: euclidean_precision - value: 75.7128235122094 - - type: euclidean_recall - value: 79.73514012935017 - - type: manhattan_accuracy - value: 88.62692591298949 - - type: manhattan_ap - value: 84.80451408255276 - - type: manhattan_f1 - value: 77.69888949572183 - - type: manhattan_precision - value: 73.70311528631622 - - type: manhattan_recall - value: 82.15275639051433 - - type: max_accuracy - value: 88.6987231730508 - - type: max_ap - value: 84.82526278228542 - - type: max_f1 - value: 77.69888949572183 - task: - type: PairClassification - - dataset: - config: ru-en - name: MTEB BUCC.v2 (ru-en) - revision: 1739dc11ffe9b7bfccd7f3d585aeb4c544fc6677 - split: test - type: mteb/bucc-bitext-mining - metrics: - - type: accuracy - value: 95.72566678212678 - - type: f1 - value: 94.42443135896548 - - type: main_score - value: 94.42443135896548 - - type: precision - value: 93.80868260016165 - - type: recall - value: 95.72566678212678 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-rus_Cyrl - name: MTEB BelebeleRetrieval (rus_Cyrl-rus_Cyrl) - revision: 75b399394a9803252cfec289d103de462763db7c - split: test - type: facebook/belebele - metrics: - - type: main_score - value: 92.23599999999999 - - type: map_at_1 - value: 87.111 - - type: map_at_10 - value: 90.717 - - type: map_at_100 - value: 90.879 - - type: map_at_1000 - value: 90.881 - - type: map_at_20 - value: 90.849 - - type: map_at_3 - value: 90.074 - - type: map_at_5 - value: 90.535 - - type: mrr_at_1 - value: 87.1111111111111 - - type: mrr_at_10 - value: 90.7173721340388 - - type: mrr_at_100 - value: 90.87859682638407 - - type: mrr_at_1000 - value: 90.88093553612326 - - type: mrr_at_20 - value: 90.84863516113515 - - type: mrr_at_3 - value: 90.07407407407409 - - type: mrr_at_5 - value: 90.53518518518521 - - type: nauc_map_at_1000_diff1 - value: 92.37373187280554 - - type: nauc_map_at_1000_max - value: 79.90465445423249 - - type: nauc_map_at_1000_std - value: -0.6220290556185463 - - type: nauc_map_at_100_diff1 - value: 92.37386697345335 - - type: nauc_map_at_100_max - value: 79.90991577223959 - - type: nauc_map_at_100_std - value: -0.602247514642845 - - type: nauc_map_at_10_diff1 - value: 92.30907447072467 - - type: nauc_map_at_10_max - value: 79.86831935337598 - - type: nauc_map_at_10_std - value: -0.7455191860719699 - - type: nauc_map_at_1_diff1 - value: 93.29828518358822 - - type: nauc_map_at_1_max - value: 78.69539619887887 - - type: nauc_map_at_1_std - value: -4.097150817605763 - - type: nauc_map_at_20_diff1 - value: 92.38414149703077 - - type: nauc_map_at_20_max - value: 79.94789814504661 - - type: nauc_map_at_20_std - value: -0.3928031130400773 - - type: nauc_map_at_3_diff1 - value: 92.21688899306734 - - type: nauc_map_at_3_max - value: 80.34586671780885 - - type: nauc_map_at_3_std - value: 0.24088319695435909 - - type: nauc_map_at_5_diff1 - value: 92.27931726042982 - - type: nauc_map_at_5_max - value: 79.99198834003367 - - type: nauc_map_at_5_std - value: -0.6296366922840796 - - type: nauc_mrr_at_1000_diff1 - value: 92.37373187280554 - - type: nauc_mrr_at_1000_max - value: 79.90465445423249 - - type: nauc_mrr_at_1000_std - value: -0.6220290556185463 - - type: nauc_mrr_at_100_diff1 - value: 92.37386697345335 - - type: nauc_mrr_at_100_max - value: 79.90991577223959 - - type: nauc_mrr_at_100_std - value: -0.602247514642845 - - type: nauc_mrr_at_10_diff1 - value: 92.30907447072467 - - type: nauc_mrr_at_10_max - value: 79.86831935337598 - - type: nauc_mrr_at_10_std - value: -0.7455191860719699 - - type: nauc_mrr_at_1_diff1 - value: 93.29828518358822 - - type: nauc_mrr_at_1_max - value: 78.69539619887887 - - type: nauc_mrr_at_1_std - value: -4.097150817605763 - - type: nauc_mrr_at_20_diff1 - value: 92.38414149703077 - - type: nauc_mrr_at_20_max - value: 79.94789814504661 - - type: nauc_mrr_at_20_std - value: -0.3928031130400773 - - type: nauc_mrr_at_3_diff1 - value: 92.21688899306734 - - type: nauc_mrr_at_3_max - value: 80.34586671780885 - - type: nauc_mrr_at_3_std - value: 0.24088319695435909 - - type: nauc_mrr_at_5_diff1 - value: 92.27931726042982 - - type: nauc_mrr_at_5_max - value: 79.99198834003367 - - type: nauc_mrr_at_5_std - value: -0.6296366922840796 - - type: nauc_ndcg_at_1000_diff1 - value: 92.30526497646306 - - type: nauc_ndcg_at_1000_max - value: 80.12734537480418 - - type: nauc_ndcg_at_1000_std - value: 0.22849408935578744 - - type: nauc_ndcg_at_100_diff1 - value: 92.31347123202318 - - type: nauc_ndcg_at_100_max - value: 80.29207038703142 - - type: nauc_ndcg_at_100_std - value: 0.816825944406239 - - type: nauc_ndcg_at_10_diff1 - value: 92.05430189845808 - - type: nauc_ndcg_at_10_max - value: 80.16515667442968 - - type: nauc_ndcg_at_10_std - value: 0.7486447532544893 - - type: nauc_ndcg_at_1_diff1 - value: 93.29828518358822 - - type: nauc_ndcg_at_1_max - value: 78.69539619887887 - - type: nauc_ndcg_at_1_std - value: -4.097150817605763 - - type: nauc_ndcg_at_20_diff1 - value: 92.40147868825079 - - type: nauc_ndcg_at_20_max - value: 80.5117307181802 - - type: nauc_ndcg_at_20_std - value: 2.0431351539517033 - - type: nauc_ndcg_at_3_diff1 - value: 91.88894444422789 - - type: nauc_ndcg_at_3_max - value: 81.09256084196045 - - type: nauc_ndcg_at_3_std - value: 2.422705909643621 - - type: nauc_ndcg_at_5_diff1 - value: 91.99711052955728 - - type: nauc_ndcg_at_5_max - value: 80.46996334573979 - - type: nauc_ndcg_at_5_std - value: 0.9086986899040708 - - type: nauc_precision_at_1000_diff1 - value: .nan - - type: nauc_precision_at_1000_max - value: .nan - - type: nauc_precision_at_1000_std - value: .nan - - type: nauc_precision_at_100_diff1 - value: 93.46405228758012 - - type: nauc_precision_at_100_max - value: 100.0 - - type: nauc_precision_at_100_std - value: 70.71661998132774 - - type: nauc_precision_at_10_diff1 - value: 90.13938908896874 - - type: nauc_precision_at_10_max - value: 82.21121782046167 - - type: nauc_precision_at_10_std - value: 13.075230092036083 - - type: nauc_precision_at_1_diff1 - value: 93.29828518358822 - - type: nauc_precision_at_1_max - value: 78.69539619887887 - - type: nauc_precision_at_1_std - value: -4.097150817605763 - - type: nauc_precision_at_20_diff1 - value: 94.9723479135242 - - type: nauc_precision_at_20_max - value: 91.04000574588684 - - type: nauc_precision_at_20_std - value: 48.764634058749586 - - type: nauc_precision_at_3_diff1 - value: 90.52690041533852 - - type: nauc_precision_at_3_max - value: 84.35075179497126 - - type: nauc_precision_at_3_std - value: 12.036768730480507 - - type: nauc_precision_at_5_diff1 - value: 90.44234360410769 - - type: nauc_precision_at_5_max - value: 83.21895424836558 - - type: nauc_precision_at_5_std - value: 9.974323062558037 - - type: nauc_recall_at_1000_diff1 - value: .nan - - type: nauc_recall_at_1000_max - value: .nan - - type: nauc_recall_at_1000_std - value: .nan - - type: nauc_recall_at_100_diff1 - value: 93.46405228758294 - - type: nauc_recall_at_100_max - value: 100.0 - - type: nauc_recall_at_100_std - value: 70.71661998132666 - - type: nauc_recall_at_10_diff1 - value: 90.13938908896864 - - type: nauc_recall_at_10_max - value: 82.21121782046124 - - type: nauc_recall_at_10_std - value: 13.075230092036506 - - type: nauc_recall_at_1_diff1 - value: 93.29828518358822 - - type: nauc_recall_at_1_max - value: 78.69539619887887 - - type: nauc_recall_at_1_std - value: -4.097150817605763 - - type: nauc_recall_at_20_diff1 - value: 94.97234791352489 - - type: nauc_recall_at_20_max - value: 91.04000574588774 - - type: nauc_recall_at_20_std - value: 48.764634058752065 - - type: nauc_recall_at_3_diff1 - value: 90.52690041533845 - - type: nauc_recall_at_3_max - value: 84.35075179497079 - - type: nauc_recall_at_3_std - value: 12.036768730480583 - - type: nauc_recall_at_5_diff1 - value: 90.44234360410861 - - type: nauc_recall_at_5_max - value: 83.21895424836595 - - type: nauc_recall_at_5_std - value: 9.974323062558147 - - type: ndcg_at_1 - value: 87.111 - - type: ndcg_at_10 - value: 92.23599999999999 - - type: ndcg_at_100 - value: 92.87100000000001 - - type: ndcg_at_1000 - value: 92.928 - - type: ndcg_at_20 - value: 92.67699999999999 - - type: ndcg_at_3 - value: 90.973 - - type: ndcg_at_5 - value: 91.801 - - type: precision_at_1 - value: 87.111 - - type: precision_at_10 - value: 9.689 - - type: precision_at_100 - value: 0.996 - - type: precision_at_1000 - value: 0.1 - - type: precision_at_20 - value: 4.928 - - type: precision_at_3 - value: 31.185000000000002 - - type: precision_at_5 - value: 19.111 - - type: recall_at_1 - value: 87.111 - - type: recall_at_10 - value: 96.88900000000001 - - type: recall_at_100 - value: 99.556 - - type: recall_at_1000 - value: 100.0 - - type: recall_at_20 - value: 98.556 - - type: recall_at_3 - value: 93.556 - - type: recall_at_5 - value: 95.556 - task: - type: Retrieval - - dataset: - config: rus_Cyrl-eng_Latn - name: MTEB BelebeleRetrieval (rus_Cyrl-eng_Latn) - revision: 75b399394a9803252cfec289d103de462763db7c - split: test - type: facebook/belebele - metrics: - - type: main_score - value: 86.615 - - type: map_at_1 - value: 78.0 - - type: map_at_10 - value: 83.822 - - type: map_at_100 - value: 84.033 - - type: map_at_1000 - value: 84.03500000000001 - - type: map_at_20 - value: 83.967 - - type: map_at_3 - value: 82.315 - - type: map_at_5 - value: 83.337 - - type: mrr_at_1 - value: 78.0 - - type: mrr_at_10 - value: 83.82213403880073 - - type: mrr_at_100 - value: 84.03281327810801 - - type: mrr_at_1000 - value: 84.03460051000452 - - type: mrr_at_20 - value: 83.9673773122303 - - type: mrr_at_3 - value: 82.31481481481484 - - type: mrr_at_5 - value: 83.33703703703708 - - type: nauc_map_at_1000_diff1 - value: 80.78467576987832 - - type: nauc_map_at_1000_max - value: 51.41718334647604 - - type: nauc_map_at_1000_std - value: -16.23873782768812 - - type: nauc_map_at_100_diff1 - value: 80.78490931240695 - - type: nauc_map_at_100_max - value: 51.41504597713061 - - type: nauc_map_at_100_std - value: -16.23538559475366 - - type: nauc_map_at_10_diff1 - value: 80.73989245374868 - - type: nauc_map_at_10_max - value: 51.43026079433827 - - type: nauc_map_at_10_std - value: -16.13414330905897 - - type: nauc_map_at_1_diff1 - value: 82.36966971144186 - - type: nauc_map_at_1_max - value: 52.988877039509916 - - type: nauc_map_at_1_std - value: -15.145824639495546 - - type: nauc_map_at_20_diff1 - value: 80.75923781626145 - - type: nauc_map_at_20_max - value: 51.40181079374639 - - type: nauc_map_at_20_std - value: -16.260566097377165 - - type: nauc_map_at_3_diff1 - value: 80.65242627065471 - - type: nauc_map_at_3_max - value: 50.623980338841214 - - type: nauc_map_at_3_std - value: -16.818343442794294 - - type: nauc_map_at_5_diff1 - value: 80.45976387021862 - - type: nauc_map_at_5_max - value: 51.533621728445866 - - type: nauc_map_at_5_std - value: -16.279891536945815 - - type: nauc_mrr_at_1000_diff1 - value: 80.78467576987832 - - type: nauc_mrr_at_1000_max - value: 51.41718334647604 - - type: nauc_mrr_at_1000_std - value: -16.23873782768812 - - type: nauc_mrr_at_100_diff1 - value: 80.78490931240695 - - type: nauc_mrr_at_100_max - value: 51.41504597713061 - - type: nauc_mrr_at_100_std - value: -16.23538559475366 - - type: nauc_mrr_at_10_diff1 - value: 80.73989245374868 - - type: nauc_mrr_at_10_max - value: 51.43026079433827 - - type: nauc_mrr_at_10_std - value: -16.13414330905897 - - type: nauc_mrr_at_1_diff1 - value: 82.36966971144186 - - type: nauc_mrr_at_1_max - value: 52.988877039509916 - - type: nauc_mrr_at_1_std - value: -15.145824639495546 - - type: nauc_mrr_at_20_diff1 - value: 80.75923781626145 - - type: nauc_mrr_at_20_max - value: 51.40181079374639 - - type: nauc_mrr_at_20_std - value: -16.260566097377165 - - type: nauc_mrr_at_3_diff1 - value: 80.65242627065471 - - type: nauc_mrr_at_3_max - value: 50.623980338841214 - - type: nauc_mrr_at_3_std - value: -16.818343442794294 - - type: nauc_mrr_at_5_diff1 - value: 80.45976387021862 - - type: nauc_mrr_at_5_max - value: 51.533621728445866 - - type: nauc_mrr_at_5_std - value: -16.279891536945815 - - type: nauc_ndcg_at_1000_diff1 - value: 80.60009446938174 - - type: nauc_ndcg_at_1000_max - value: 51.381708043594166 - - type: nauc_ndcg_at_1000_std - value: -16.054256944160848 - - type: nauc_ndcg_at_100_diff1 - value: 80.58971462930421 - - type: nauc_ndcg_at_100_max - value: 51.25436917735444 - - type: nauc_ndcg_at_100_std - value: -15.862944972269894 - - type: nauc_ndcg_at_10_diff1 - value: 80.37967179454489 - - type: nauc_ndcg_at_10_max - value: 51.590394257251006 - - type: nauc_ndcg_at_10_std - value: -15.489799384799591 - - type: nauc_ndcg_at_1_diff1 - value: 82.36966971144186 - - type: nauc_ndcg_at_1_max - value: 52.988877039509916 - - type: nauc_ndcg_at_1_std - value: -15.145824639495546 - - type: nauc_ndcg_at_20_diff1 - value: 80.40299527470081 - - type: nauc_ndcg_at_20_max - value: 51.395132284307074 - - type: nauc_ndcg_at_20_std - value: -15.906165526937203 - - type: nauc_ndcg_at_3_diff1 - value: 80.10347913649302 - - type: nauc_ndcg_at_3_max - value: 50.018431855573844 - - type: nauc_ndcg_at_3_std - value: -17.12743750163884 - - type: nauc_ndcg_at_5_diff1 - value: 79.65918647776613 - - type: nauc_ndcg_at_5_max - value: 51.76710880330806 - - type: nauc_ndcg_at_5_std - value: -16.071901882035945 - - type: nauc_precision_at_1000_diff1 - value: .nan - - type: nauc_precision_at_1000_max - value: .nan - - type: nauc_precision_at_1000_std - value: .nan - - type: nauc_precision_at_100_diff1 - value: 77.41596638655459 - - type: nauc_precision_at_100_max - value: 22.572362278246565 - - type: nauc_precision_at_100_std - value: 26.890756302525716 - - type: nauc_precision_at_10_diff1 - value: 77.82112845138009 - - type: nauc_precision_at_10_max - value: 54.2550353474723 - - type: nauc_precision_at_10_std - value: -7.492997198879646 - - type: nauc_precision_at_1_diff1 - value: 82.36966971144186 - - type: nauc_precision_at_1_max - value: 52.988877039509916 - - type: nauc_precision_at_1_std - value: -15.145824639495546 - - type: nauc_precision_at_20_diff1 - value: 75.89091192032318 - - type: nauc_precision_at_20_max - value: 52.03275754746293 - - type: nauc_precision_at_20_std - value: -7.8411920323686175 - - type: nauc_precision_at_3_diff1 - value: 78.0256020644638 - - type: nauc_precision_at_3_max - value: 47.80353641248523 - - type: nauc_precision_at_3_std - value: -18.181625255723503 - - type: nauc_precision_at_5_diff1 - value: 75.21583976056174 - - type: nauc_precision_at_5_max - value: 53.716281032960765 - - type: nauc_precision_at_5_std - value: -14.411700753360812 - - type: nauc_recall_at_1000_diff1 - value: .nan - - type: nauc_recall_at_1000_max - value: .nan - - type: nauc_recall_at_1000_std - value: .nan - - type: nauc_recall_at_100_diff1 - value: 77.4159663865523 - - type: nauc_recall_at_100_max - value: 22.57236227824646 - - type: nauc_recall_at_100_std - value: 26.89075630252133 - - type: nauc_recall_at_10_diff1 - value: 77.82112845138037 - - type: nauc_recall_at_10_max - value: 54.25503534747204 - - type: nauc_recall_at_10_std - value: -7.492997198879666 - - type: nauc_recall_at_1_diff1 - value: 82.36966971144186 - - type: nauc_recall_at_1_max - value: 52.988877039509916 - - type: nauc_recall_at_1_std - value: -15.145824639495546 - - type: nauc_recall_at_20_diff1 - value: 75.89091192032362 - - type: nauc_recall_at_20_max - value: 52.032757547463184 - - type: nauc_recall_at_20_std - value: -7.84119203236888 - - type: nauc_recall_at_3_diff1 - value: 78.02560206446354 - - type: nauc_recall_at_3_max - value: 47.80353641248526 - - type: nauc_recall_at_3_std - value: -18.181625255723656 - - type: nauc_recall_at_5_diff1 - value: 75.21583976056185 - - type: nauc_recall_at_5_max - value: 53.71628103296118 - - type: nauc_recall_at_5_std - value: -14.411700753360634 - - type: ndcg_at_1 - value: 78.0 - - type: ndcg_at_10 - value: 86.615 - - type: ndcg_at_100 - value: 87.558 - - type: ndcg_at_1000 - value: 87.613 - - type: ndcg_at_20 - value: 87.128 - - type: ndcg_at_3 - value: 83.639 - - type: ndcg_at_5 - value: 85.475 - - type: precision_at_1 - value: 78.0 - - type: precision_at_10 - value: 9.533 - - type: precision_at_100 - value: 0.996 - - type: precision_at_1000 - value: 0.1 - - type: precision_at_20 - value: 4.867 - - type: precision_at_3 - value: 29.148000000000003 - - type: precision_at_5 - value: 18.378 - - type: recall_at_1 - value: 78.0 - - type: recall_at_10 - value: 95.333 - - type: recall_at_100 - value: 99.556 - - type: recall_at_1000 - value: 100.0 - - type: recall_at_20 - value: 97.333 - - type: recall_at_3 - value: 87.444 - - type: recall_at_5 - value: 91.889 - task: - type: Retrieval - - dataset: - config: eng_Latn-rus_Cyrl - name: MTEB BelebeleRetrieval (eng_Latn-rus_Cyrl) - revision: 75b399394a9803252cfec289d103de462763db7c - split: test - type: facebook/belebele - metrics: - - type: main_score - value: 82.748 - - type: map_at_1 - value: 73.444 - - type: map_at_10 - value: 79.857 - - type: map_at_100 - value: 80.219 - - type: map_at_1000 - value: 80.22500000000001 - - type: map_at_20 - value: 80.10300000000001 - - type: map_at_3 - value: 78.593 - - type: map_at_5 - value: 79.515 - - type: mrr_at_1 - value: 73.44444444444444 - - type: mrr_at_10 - value: 79.85705467372136 - - type: mrr_at_100 - value: 80.21942320422542 - - type: mrr_at_1000 - value: 80.2245364027152 - - type: mrr_at_20 - value: 80.10273201266493 - - type: mrr_at_3 - value: 78.59259259259258 - - type: mrr_at_5 - value: 79.51481481481483 - - type: nauc_map_at_1000_diff1 - value: 83.69682652271125 - - type: nauc_map_at_1000_max - value: 61.70131708044767 - - type: nauc_map_at_1000_std - value: 9.345825405274955 - - type: nauc_map_at_100_diff1 - value: 83.68924820523492 - - type: nauc_map_at_100_max - value: 61.6965735573098 - - type: nauc_map_at_100_std - value: 9.366132859525775 - - type: nauc_map_at_10_diff1 - value: 83.61802964269985 - - type: nauc_map_at_10_max - value: 61.74274476167882 - - type: nauc_map_at_10_std - value: 9.504060995819101 - - type: nauc_map_at_1_diff1 - value: 86.37079221403225 - - type: nauc_map_at_1_max - value: 61.856861655370686 - - type: nauc_map_at_1_std - value: 4.708911881992707 - - type: nauc_map_at_20_diff1 - value: 83.62920965453047 - - type: nauc_map_at_20_max - value: 61.761029350326965 - - type: nauc_map_at_20_std - value: 9.572978651118351 - - type: nauc_map_at_3_diff1 - value: 83.66665673154306 - - type: nauc_map_at_3_max - value: 61.13597610587937 - - type: nauc_map_at_3_std - value: 9.309596395240598 - - type: nauc_map_at_5_diff1 - value: 83.52307226455358 - - type: nauc_map_at_5_max - value: 61.59405758027573 - - type: nauc_map_at_5_std - value: 9.320025423287671 - - type: nauc_mrr_at_1000_diff1 - value: 83.69682652271125 - - type: nauc_mrr_at_1000_max - value: 61.70131708044767 - - type: nauc_mrr_at_1000_std - value: 9.345825405274955 - - type: nauc_mrr_at_100_diff1 - value: 83.68924820523492 - - type: nauc_mrr_at_100_max - value: 61.6965735573098 - - type: nauc_mrr_at_100_std - value: 9.366132859525775 - - type: nauc_mrr_at_10_diff1 - value: 83.61802964269985 - - type: nauc_mrr_at_10_max - value: 61.74274476167882 - - type: nauc_mrr_at_10_std - value: 9.504060995819101 - - type: nauc_mrr_at_1_diff1 - value: 86.37079221403225 - - type: nauc_mrr_at_1_max - value: 61.856861655370686 - - type: nauc_mrr_at_1_std - value: 4.708911881992707 - - type: nauc_mrr_at_20_diff1 - value: 83.62920965453047 - - type: nauc_mrr_at_20_max - value: 61.761029350326965 - - type: nauc_mrr_at_20_std - value: 9.572978651118351 - - type: nauc_mrr_at_3_diff1 - value: 83.66665673154306 - - type: nauc_mrr_at_3_max - value: 61.13597610587937 - - type: nauc_mrr_at_3_std - value: 9.309596395240598 - - type: nauc_mrr_at_5_diff1 - value: 83.52307226455358 - - type: nauc_mrr_at_5_max - value: 61.59405758027573 - - type: nauc_mrr_at_5_std - value: 9.320025423287671 - - type: nauc_ndcg_at_1000_diff1 - value: 83.24213186482201 - - type: nauc_ndcg_at_1000_max - value: 61.77629841787496 - - type: nauc_ndcg_at_1000_std - value: 10.332527869705851 - - type: nauc_ndcg_at_100_diff1 - value: 83.06815820441027 - - type: nauc_ndcg_at_100_max - value: 61.6947181864579 - - type: nauc_ndcg_at_100_std - value: 10.888922975877316 - - type: nauc_ndcg_at_10_diff1 - value: 82.58238431386295 - - type: nauc_ndcg_at_10_max - value: 62.10333663935709 - - type: nauc_ndcg_at_10_std - value: 11.746030330958174 - - type: nauc_ndcg_at_1_diff1 - value: 86.37079221403225 - - type: nauc_ndcg_at_1_max - value: 61.856861655370686 - - type: nauc_ndcg_at_1_std - value: 4.708911881992707 - - type: nauc_ndcg_at_20_diff1 - value: 82.67888324480154 - - type: nauc_ndcg_at_20_max - value: 62.28124917486516 - - type: nauc_ndcg_at_20_std - value: 12.343058917563914 - - type: nauc_ndcg_at_3_diff1 - value: 82.71277373710663 - - type: nauc_ndcg_at_3_max - value: 60.66677922989939 - - type: nauc_ndcg_at_3_std - value: 10.843633736296528 - - type: nauc_ndcg_at_5_diff1 - value: 82.34691124846786 - - type: nauc_ndcg_at_5_max - value: 61.605961382062716 - - type: nauc_ndcg_at_5_std - value: 11.129011077702602 - - type: nauc_precision_at_1000_diff1 - value: .nan - - type: nauc_precision_at_1000_max - value: .nan - - type: nauc_precision_at_1000_std - value: .nan - - type: nauc_precision_at_100_diff1 - value: 60.93103908230194 - - type: nauc_precision_at_100_max - value: 52.621048419370695 - - type: nauc_precision_at_100_std - value: 85.60090702947922 - - type: nauc_precision_at_10_diff1 - value: 76.26517273576093 - - type: nauc_precision_at_10_max - value: 65.2013694366636 - - type: nauc_precision_at_10_std - value: 26.50357920946173 - - type: nauc_precision_at_1_diff1 - value: 86.37079221403225 - - type: nauc_precision_at_1_max - value: 61.856861655370686 - - type: nauc_precision_at_1_std - value: 4.708911881992707 - - type: nauc_precision_at_20_diff1 - value: 73.47946930710295 - - type: nauc_precision_at_20_max - value: 70.19520986689217 - - type: nauc_precision_at_20_std - value: 45.93186111653967 - - type: nauc_precision_at_3_diff1 - value: 79.02026879450186 - - type: nauc_precision_at_3_max - value: 58.75074624692399 - - type: nauc_precision_at_3_std - value: 16.740684654251037 - - type: nauc_precision_at_5_diff1 - value: 76.47585662281637 - - type: nauc_precision_at_5_max - value: 61.86270922013127 - - type: nauc_precision_at_5_std - value: 20.1833625455035 - - type: nauc_recall_at_1000_diff1 - value: .nan - - type: nauc_recall_at_1000_max - value: .nan - - type: nauc_recall_at_1000_std - value: .nan - - type: nauc_recall_at_100_diff1 - value: 60.93103908229921 - - type: nauc_recall_at_100_max - value: 52.62104841936668 - - type: nauc_recall_at_100_std - value: 85.60090702947748 - - type: nauc_recall_at_10_diff1 - value: 76.26517273576097 - - type: nauc_recall_at_10_max - value: 65.20136943666347 - - type: nauc_recall_at_10_std - value: 26.50357920946174 - - type: nauc_recall_at_1_diff1 - value: 86.37079221403225 - - type: nauc_recall_at_1_max - value: 61.856861655370686 - - type: nauc_recall_at_1_std - value: 4.708911881992707 - - type: nauc_recall_at_20_diff1 - value: 73.47946930710269 - - type: nauc_recall_at_20_max - value: 70.19520986689254 - - type: nauc_recall_at_20_std - value: 45.93186111653943 - - type: nauc_recall_at_3_diff1 - value: 79.02026879450173 - - type: nauc_recall_at_3_max - value: 58.750746246923924 - - type: nauc_recall_at_3_std - value: 16.740684654251076 - - type: nauc_recall_at_5_diff1 - value: 76.4758566228162 - - type: nauc_recall_at_5_max - value: 61.862709220131386 - - type: nauc_recall_at_5_std - value: 20.18336254550361 - - type: ndcg_at_1 - value: 73.444 - - type: ndcg_at_10 - value: 82.748 - - type: ndcg_at_100 - value: 84.416 - - type: ndcg_at_1000 - value: 84.52300000000001 - - type: ndcg_at_20 - value: 83.646 - - type: ndcg_at_3 - value: 80.267 - - type: ndcg_at_5 - value: 81.922 - - type: precision_at_1 - value: 73.444 - - type: precision_at_10 - value: 9.167 - - type: precision_at_100 - value: 0.992 - - type: precision_at_1000 - value: 0.1 - - type: precision_at_20 - value: 4.761 - - type: precision_at_3 - value: 28.37 - - type: precision_at_5 - value: 17.822 - - type: recall_at_1 - value: 73.444 - - type: recall_at_10 - value: 91.667 - - type: recall_at_100 - value: 99.222 - - type: recall_at_1000 - value: 100.0 - - type: recall_at_20 - value: 95.222 - - type: recall_at_3 - value: 85.111 - - type: recall_at_5 - value: 89.11099999999999 - task: - type: Retrieval - - dataset: - config: eng_Latn-rus_Cyrl - name: MTEB BibleNLPBitextMining (eng_Latn-rus_Cyrl) - revision: 264a18480c529d9e922483839b4b9758e690b762 - split: train - type: davidstap/biblenlp-corpus-mmteb - metrics: - - type: accuracy - value: 96.875 - - type: f1 - value: 95.83333333333333 - - type: main_score - value: 95.83333333333333 - - type: precision - value: 95.3125 - - type: recall - value: 96.875 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-eng_Latn - name: MTEB BibleNLPBitextMining (rus_Cyrl-eng_Latn) - revision: 264a18480c529d9e922483839b4b9758e690b762 - split: train - type: davidstap/biblenlp-corpus-mmteb - metrics: - - type: accuracy - value: 88.671875 - - type: f1 - value: 85.3515625 - - type: main_score - value: 85.3515625 - - type: precision - value: 83.85416666666667 - - type: recall - value: 88.671875 - task: - type: BitextMining - - dataset: - config: default - name: MTEB CEDRClassification (default) - revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4 - split: test - type: ai-forever/cedr-classification - metrics: - - type: accuracy - value: 40.06907545164719 - - type: f1 - value: 26.285000550712407 - - type: lrap - value: 64.4280021253997 - - type: main_score - value: 40.06907545164719 - task: - type: MultilabelClassification - - dataset: - config: default - name: MTEB CyrillicTurkicLangClassification (default) - revision: e42d330f33d65b7b72dfd408883daf1661f06f18 - split: test - type: tatiana-merz/cyrillic_turkic_langs - metrics: - - type: accuracy - value: 43.3447265625 - - type: f1 - value: 40.08400146827895 - - type: f1_weighted - value: 40.08499428040896 - - type: main_score - value: 43.3447265625 - task: - type: Classification - - dataset: - config: ace_Arab-rus_Cyrl - name: MTEB FloresBitextMining (ace_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 6.225296442687747 - - type: f1 - value: 5.5190958860075 - - type: main_score - value: 5.5190958860075 - - type: precision - value: 5.3752643758000005 - - type: recall - value: 6.225296442687747 - task: - type: BitextMining - - dataset: - config: bam_Latn-rus_Cyrl - name: MTEB FloresBitextMining (bam_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 68.37944664031622 - - type: f1 - value: 64.54819836666252 - - type: main_score - value: 64.54819836666252 - - type: precision - value: 63.07479233454916 - - type: recall - value: 68.37944664031622 - task: - type: BitextMining - - dataset: - config: dzo_Tibt-rus_Cyrl - name: MTEB FloresBitextMining (dzo_Tibt-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 0.09881422924901186 - - type: f1 - value: 0.00019509225912934226 - - type: main_score - value: 0.00019509225912934226 - - type: precision - value: 9.76425190207627e-05 - - type: recall - value: 0.09881422924901186 - task: - type: BitextMining - - dataset: - config: hin_Deva-rus_Cyrl - name: MTEB FloresBitextMining (hin_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.60474308300395 - - type: f1 - value: 99.47299077733861 - - type: main_score - value: 99.47299077733861 - - type: precision - value: 99.40711462450594 - - type: recall - value: 99.60474308300395 - task: - type: BitextMining - - dataset: - config: khm_Khmr-rus_Cyrl - name: MTEB FloresBitextMining (khm_Khmr-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 88.83399209486166 - - type: f1 - value: 87.71151056318254 - - type: main_score - value: 87.71151056318254 - - type: precision - value: 87.32012500709193 - - type: recall - value: 88.83399209486166 - task: - type: BitextMining - - dataset: - config: mag_Deva-rus_Cyrl - name: MTEB FloresBitextMining (mag_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.7239789196311 - - type: main_score - value: 97.7239789196311 - - type: precision - value: 97.61904761904762 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: pap_Latn-rus_Cyrl - name: MTEB FloresBitextMining (pap_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.0711462450593 - - type: f1 - value: 93.68187806922984 - - type: main_score - value: 93.68187806922984 - - type: precision - value: 93.58925452707051 - - type: recall - value: 94.0711462450593 - task: - type: BitextMining - - dataset: - config: sot_Latn-rus_Cyrl - name: MTEB FloresBitextMining (sot_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 90.9090909090909 - - type: f1 - value: 89.23171936758892 - - type: main_score - value: 89.23171936758892 - - type: precision - value: 88.51790014083866 - - type: recall - value: 90.9090909090909 - task: - type: BitextMining - - dataset: - config: tur_Latn-rus_Cyrl - name: MTEB FloresBitextMining (tur_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.9459815546772 - - type: main_score - value: 98.9459815546772 - - type: precision - value: 98.81422924901186 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: ace_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ace_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 66.10671936758892 - - type: f1 - value: 63.81888256297873 - - type: main_score - value: 63.81888256297873 - - type: precision - value: 63.01614067933451 - - type: recall - value: 66.10671936758892 - task: - type: BitextMining - - dataset: - config: ban_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ban_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 79.44664031620553 - - type: f1 - value: 77.6311962082713 - - type: main_score - value: 77.6311962082713 - - type: precision - value: 76.93977931929739 - - type: recall - value: 79.44664031620553 - task: - type: BitextMining - - dataset: - config: ell_Grek-rus_Cyrl - name: MTEB FloresBitextMining (ell_Grek-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.2094861660079 - - type: main_score - value: 99.2094861660079 - - type: precision - value: 99.1106719367589 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: hne_Deva-rus_Cyrl - name: MTEB FloresBitextMining (hne_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.83794466403161 - - type: f1 - value: 96.25352907961603 - - type: main_score - value: 96.25352907961603 - - type: precision - value: 96.02155091285526 - - type: recall - value: 96.83794466403161 - task: - type: BitextMining - - dataset: - config: kik_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kik_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 76.28458498023716 - - type: f1 - value: 73.5596919895859 - - type: main_score - value: 73.5596919895859 - - type: precision - value: 72.40900759055246 - - type: recall - value: 76.28458498023716 - task: - type: BitextMining - - dataset: - config: mai_Deva-rus_Cyrl - name: MTEB FloresBitextMining (mai_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.72727272727273 - - type: f1 - value: 97.37812911725956 - - type: main_score - value: 97.37812911725956 - - type: precision - value: 97.26002258610953 - - type: recall - value: 97.72727272727273 - task: - type: BitextMining - - dataset: - config: pbt_Arab-rus_Cyrl - name: MTEB FloresBitextMining (pbt_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.0711462450593 - - type: f1 - value: 93.34700387331966 - - type: main_score - value: 93.34700387331966 - - type: precision - value: 93.06920556920556 - - type: recall - value: 94.0711462450593 - task: - type: BitextMining - - dataset: - config: spa_Latn-rus_Cyrl - name: MTEB FloresBitextMining (spa_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.9459815546772 - - type: main_score - value: 98.9459815546772 - - type: precision - value: 98.81422924901186 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: twi_Latn-rus_Cyrl - name: MTEB FloresBitextMining (twi_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 80.73122529644269 - - type: f1 - value: 77.77434363246721 - - type: main_score - value: 77.77434363246721 - - type: precision - value: 76.54444287596462 - - type: recall - value: 80.73122529644269 - task: - type: BitextMining - - dataset: - config: acm_Arab-rus_Cyrl - name: MTEB FloresBitextMining (acm_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.56521739130434 - - type: f1 - value: 92.92490118577075 - - type: main_score - value: 92.92490118577075 - - type: precision - value: 92.16897233201581 - - type: recall - value: 94.56521739130434 - task: - type: BitextMining - - dataset: - config: bel_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (bel_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.98550724637681 - - type: main_score - value: 98.98550724637681 - - type: precision - value: 98.88833992094862 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: eng_Latn-rus_Cyrl - name: MTEB FloresBitextMining (eng_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.60474308300395 - - type: f1 - value: 99.4729907773386 - - type: main_score - value: 99.4729907773386 - - type: precision - value: 99.40711462450594 - - type: recall - value: 99.60474308300395 - task: - type: BitextMining - - dataset: - config: hrv_Latn-rus_Cyrl - name: MTEB FloresBitextMining (hrv_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 99.05138339920948 - - type: main_score - value: 99.05138339920948 - - type: precision - value: 99.00691699604744 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: kin_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kin_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 88.2411067193676 - - type: f1 - value: 86.5485246227658 - - type: main_score - value: 86.5485246227658 - - type: precision - value: 85.90652101521667 - - type: recall - value: 88.2411067193676 - task: - type: BitextMining - - dataset: - config: mal_Mlym-rus_Cyrl - name: MTEB FloresBitextMining (mal_Mlym-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.51778656126481 - - type: f1 - value: 98.07971014492753 - - type: main_score - value: 98.07971014492753 - - type: precision - value: 97.88372859025033 - - type: recall - value: 98.51778656126481 - task: - type: BitextMining - - dataset: - config: pes_Arab-rus_Cyrl - name: MTEB FloresBitextMining (pes_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.51778656126481 - - type: f1 - value: 98.0566534914361 - - type: main_score - value: 98.0566534914361 - - type: precision - value: 97.82608695652173 - - type: recall - value: 98.51778656126481 - task: - type: BitextMining - - dataset: - config: srd_Latn-rus_Cyrl - name: MTEB FloresBitextMining (srd_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 82.6086956521739 - - type: f1 - value: 80.9173470979821 - - type: main_score - value: 80.9173470979821 - - type: precision - value: 80.24468672882627 - - type: recall - value: 82.6086956521739 - task: - type: BitextMining - - dataset: - config: tzm_Tfng-rus_Cyrl - name: MTEB FloresBitextMining (tzm_Tfng-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 7.41106719367589 - - type: f1 - value: 6.363562740945329 - - type: main_score - value: 6.363562740945329 - - type: precision - value: 6.090373175353411 - - type: recall - value: 7.41106719367589 - task: - type: BitextMining - - dataset: - config: acq_Arab-rus_Cyrl - name: MTEB FloresBitextMining (acq_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.25691699604744 - - type: f1 - value: 93.81422924901187 - - type: main_score - value: 93.81422924901187 - - type: precision - value: 93.14064558629775 - - type: recall - value: 95.25691699604744 - task: - type: BitextMining - - dataset: - config: bem_Latn-rus_Cyrl - name: MTEB FloresBitextMining (bem_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 68.08300395256917 - - type: f1 - value: 65.01368772860867 - - type: main_score - value: 65.01368772860867 - - type: precision - value: 63.91052337510628 - - type: recall - value: 68.08300395256917 - task: - type: BitextMining - - dataset: - config: epo_Latn-rus_Cyrl - name: MTEB FloresBitextMining (epo_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.41897233201581 - - type: f1 - value: 98.17193675889328 - - type: main_score - value: 98.17193675889328 - - type: precision - value: 98.08210564139418 - - type: recall - value: 98.41897233201581 - task: - type: BitextMining - - dataset: - config: hun_Latn-rus_Cyrl - name: MTEB FloresBitextMining (hun_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.1106719367589 - - type: main_score - value: 99.1106719367589 - - type: precision - value: 99.01185770750988 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: kir_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (kir_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.5296442687747 - - type: f1 - value: 97.07549806364035 - - type: main_score - value: 97.07549806364035 - - type: precision - value: 96.90958498023716 - - type: recall - value: 97.5296442687747 - task: - type: BitextMining - - dataset: - config: mar_Deva-rus_Cyrl - name: MTEB FloresBitextMining (mar_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.82608695652173 - - type: f1 - value: 97.44400527009222 - - type: main_score - value: 97.44400527009222 - - type: precision - value: 97.28966685488425 - - type: recall - value: 97.82608695652173 - task: - type: BitextMining - - dataset: - config: plt_Latn-rus_Cyrl - name: MTEB FloresBitextMining (plt_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 79.9407114624506 - - type: f1 - value: 78.3154177760691 - - type: main_score - value: 78.3154177760691 - - type: precision - value: 77.69877344877344 - - type: recall - value: 79.9407114624506 - task: - type: BitextMining - - dataset: - config: srp_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (srp_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.70355731225297 - - type: f1 - value: 99.60474308300395 - - type: main_score - value: 99.60474308300395 - - type: precision - value: 99.55533596837944 - - type: recall - value: 99.70355731225297 - task: - type: BitextMining - - dataset: - config: uig_Arab-rus_Cyrl - name: MTEB FloresBitextMining (uig_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 83.20158102766798 - - type: f1 - value: 81.44381923034585 - - type: main_score - value: 81.44381923034585 - - type: precision - value: 80.78813411582477 - - type: recall - value: 83.20158102766798 - task: - type: BitextMining - - dataset: - config: aeb_Arab-rus_Cyrl - name: MTEB FloresBitextMining (aeb_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.20553359683794 - - type: f1 - value: 88.75352907961603 - - type: main_score - value: 88.75352907961603 - - type: precision - value: 87.64328063241106 - - type: recall - value: 91.20553359683794 - task: - type: BitextMining - - dataset: - config: ben_Beng-rus_Cyrl - name: MTEB FloresBitextMining (ben_Beng-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.60671936758894 - - type: main_score - value: 98.60671936758894 - - type: precision - value: 98.4766139657444 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: est_Latn-rus_Cyrl - name: MTEB FloresBitextMining (est_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.24505928853755 - - type: f1 - value: 95.27417027417027 - - type: main_score - value: 95.27417027417027 - - type: precision - value: 94.84107378129117 - - type: recall - value: 96.24505928853755 - task: - type: BitextMining - - dataset: - config: hye_Armn-rus_Cyrl - name: MTEB FloresBitextMining (hye_Armn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.67786561264822 - - type: main_score - value: 97.67786561264822 - - type: precision - value: 97.55839022637441 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: kmb_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kmb_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 46.047430830039524 - - type: f1 - value: 42.94464804804471 - - type: main_score - value: 42.94464804804471 - - type: precision - value: 41.9851895607238 - - type: recall - value: 46.047430830039524 - task: - type: BitextMining - - dataset: - config: min_Arab-rus_Cyrl - name: MTEB FloresBitextMining (min_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 3.9525691699604746 - - type: f1 - value: 3.402665192725756 - - type: main_score - value: 3.402665192725756 - - type: precision - value: 3.303787557740127 - - type: recall - value: 3.9525691699604746 - task: - type: BitextMining - - dataset: - config: pol_Latn-rus_Cyrl - name: MTEB FloresBitextMining (pol_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.60474308300395 - - type: f1 - value: 99.4729907773386 - - type: main_score - value: 99.4729907773386 - - type: precision - value: 99.40711462450594 - - type: recall - value: 99.60474308300395 - task: - type: BitextMining - - dataset: - config: ssw_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ssw_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 73.22134387351778 - - type: f1 - value: 70.43086049508975 - - type: main_score - value: 70.43086049508975 - - type: precision - value: 69.35312022355656 - - type: recall - value: 73.22134387351778 - task: - type: BitextMining - - dataset: - config: ukr_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (ukr_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.90118577075098 - - type: f1 - value: 99.86824769433464 - - type: main_score - value: 99.86824769433464 - - type: precision - value: 99.85177865612648 - - type: recall - value: 99.90118577075098 - task: - type: BitextMining - - dataset: - config: afr_Latn-rus_Cyrl - name: MTEB FloresBitextMining (afr_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.9459815546772 - - type: main_score - value: 98.9459815546772 - - type: precision - value: 98.81422924901186 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: bho_Deva-rus_Cyrl - name: MTEB FloresBitextMining (bho_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.0711462450593 - - type: f1 - value: 93.12182382834557 - - type: main_score - value: 93.12182382834557 - - type: precision - value: 92.7523453232338 - - type: recall - value: 94.0711462450593 - task: - type: BitextMining - - dataset: - config: eus_Latn-rus_Cyrl - name: MTEB FloresBitextMining (eus_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.19367588932806 - - type: f1 - value: 91.23604975587072 - - type: main_score - value: 91.23604975587072 - - type: precision - value: 90.86697443588663 - - type: recall - value: 92.19367588932806 - task: - type: BitextMining - - dataset: - config: ibo_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ibo_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 82.21343873517787 - - type: f1 - value: 80.17901604858126 - - type: main_score - value: 80.17901604858126 - - type: precision - value: 79.3792284780028 - - type: recall - value: 82.21343873517787 - task: - type: BitextMining - - dataset: - config: kmr_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kmr_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 68.67588932806325 - - type: f1 - value: 66.72311714750278 - - type: main_score - value: 66.72311714750278 - - type: precision - value: 66.00178401554004 - - type: recall - value: 68.67588932806325 - task: - type: BitextMining - - dataset: - config: min_Latn-rus_Cyrl - name: MTEB FloresBitextMining (min_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 78.65612648221344 - - type: f1 - value: 76.26592719972166 - - type: main_score - value: 76.26592719972166 - - type: precision - value: 75.39980459997484 - - type: recall - value: 78.65612648221344 - task: - type: BitextMining - - dataset: - config: por_Latn-rus_Cyrl - name: MTEB FloresBitextMining (por_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.83794466403161 - - type: f1 - value: 95.9669678147939 - - type: main_score - value: 95.9669678147939 - - type: precision - value: 95.59453227931488 - - type: recall - value: 96.83794466403161 - task: - type: BitextMining - - dataset: - config: sun_Latn-rus_Cyrl - name: MTEB FloresBitextMining (sun_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.4901185770751 - - type: f1 - value: 91.66553983773662 - - type: main_score - value: 91.66553983773662 - - type: precision - value: 91.34530928009188 - - type: recall - value: 92.4901185770751 - task: - type: BitextMining - - dataset: - config: umb_Latn-rus_Cyrl - name: MTEB FloresBitextMining (umb_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 41.00790513833992 - - type: f1 - value: 38.21319326004483 - - type: main_score - value: 38.21319326004483 - - type: precision - value: 37.200655467675546 - - type: recall - value: 41.00790513833992 - task: - type: BitextMining - - dataset: - config: ajp_Arab-rus_Cyrl - name: MTEB FloresBitextMining (ajp_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.35573122529645 - - type: f1 - value: 93.97233201581028 - - type: main_score - value: 93.97233201581028 - - type: precision - value: 93.33333333333333 - - type: recall - value: 95.35573122529645 - task: - type: BitextMining - - dataset: - config: bjn_Arab-rus_Cyrl - name: MTEB FloresBitextMining (bjn_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 3.6561264822134385 - - type: f1 - value: 3.1071978056336484 - - type: main_score - value: 3.1071978056336484 - - type: precision - value: 3.0039741229718215 - - type: recall - value: 3.6561264822134385 - task: - type: BitextMining - - dataset: - config: ewe_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ewe_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 62.845849802371546 - - type: f1 - value: 59.82201175670472 - - type: main_score - value: 59.82201175670472 - - type: precision - value: 58.72629236362003 - - type: recall - value: 62.845849802371546 - task: - type: BitextMining - - dataset: - config: ilo_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ilo_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 83.10276679841897 - - type: f1 - value: 80.75065288987582 - - type: main_score - value: 80.75065288987582 - - type: precision - value: 79.80726451662179 - - type: recall - value: 83.10276679841897 - task: - type: BitextMining - - dataset: - config: knc_Arab-rus_Cyrl - name: MTEB FloresBitextMining (knc_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 10.079051383399209 - - type: f1 - value: 8.759282456080921 - - type: main_score - value: 8.759282456080921 - - type: precision - value: 8.474735138956142 - - type: recall - value: 10.079051383399209 - task: - type: BitextMining - - dataset: - config: mkd_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (mkd_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.55072463768116 - - type: main_score - value: 98.55072463768116 - - type: precision - value: 98.36956521739131 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: prs_Arab-rus_Cyrl - name: MTEB FloresBitextMining (prs_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.68247694334651 - - type: main_score - value: 98.68247694334651 - - type: precision - value: 98.51778656126481 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: swe_Latn-rus_Cyrl - name: MTEB FloresBitextMining (swe_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.22595520421606 - - type: main_score - value: 99.22595520421606 - - type: precision - value: 99.14361001317523 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: urd_Arab-rus_Cyrl - name: MTEB FloresBitextMining (urd_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.82608695652173 - - type: f1 - value: 97.25625823451911 - - type: main_score - value: 97.25625823451911 - - type: precision - value: 97.03063241106719 - - type: recall - value: 97.82608695652173 - task: - type: BitextMining - - dataset: - config: aka_Latn-rus_Cyrl - name: MTEB FloresBitextMining (aka_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.22529644268775 - - type: f1 - value: 77.94307687941227 - - type: main_score - value: 77.94307687941227 - - type: precision - value: 76.58782793293665 - - type: recall - value: 81.22529644268775 - task: - type: BitextMining - - dataset: - config: bjn_Latn-rus_Cyrl - name: MTEB FloresBitextMining (bjn_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 85.27667984189723 - - type: f1 - value: 83.6869192829922 - - type: main_score - value: 83.6869192829922 - - type: precision - value: 83.08670670691656 - - type: recall - value: 85.27667984189723 - task: - type: BitextMining - - dataset: - config: fao_Latn-rus_Cyrl - name: MTEB FloresBitextMining (fao_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 80.9288537549407 - - type: f1 - value: 79.29806087454745 - - type: main_score - value: 79.29806087454745 - - type: precision - value: 78.71445871526987 - - type: recall - value: 80.9288537549407 - task: - type: BitextMining - - dataset: - config: ind_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ind_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.12252964426878 - - type: f1 - value: 97.5296442687747 - - type: main_score - value: 97.5296442687747 - - type: precision - value: 97.23320158102767 - - type: recall - value: 98.12252964426878 - task: - type: BitextMining - - dataset: - config: knc_Latn-rus_Cyrl - name: MTEB FloresBitextMining (knc_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 33.49802371541502 - - type: f1 - value: 32.02378215033989 - - type: main_score - value: 32.02378215033989 - - type: precision - value: 31.511356103747406 - - type: recall - value: 33.49802371541502 - task: - type: BitextMining - - dataset: - config: mlt_Latn-rus_Cyrl - name: MTEB FloresBitextMining (mlt_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.40316205533597 - - type: f1 - value: 90.35317684386006 - - type: main_score - value: 90.35317684386006 - - type: precision - value: 89.94845939633488 - - type: recall - value: 91.40316205533597 - task: - type: BitextMining - - dataset: - config: quy_Latn-rus_Cyrl - name: MTEB FloresBitextMining (quy_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 40.612648221343875 - - type: f1 - value: 38.74337544712602 - - type: main_score - value: 38.74337544712602 - - type: precision - value: 38.133716022178575 - - type: recall - value: 40.612648221343875 - task: - type: BitextMining - - dataset: - config: swh_Latn-rus_Cyrl - name: MTEB FloresBitextMining (swh_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.13438735177866 - - type: f1 - value: 96.47435897435898 - - type: main_score - value: 96.47435897435898 - - type: precision - value: 96.18741765480895 - - type: recall - value: 97.13438735177866 - task: - type: BitextMining - - dataset: - config: uzn_Latn-rus_Cyrl - name: MTEB FloresBitextMining (uzn_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.83794466403161 - - type: f1 - value: 96.26355528529442 - - type: main_score - value: 96.26355528529442 - - type: precision - value: 96.0501756697409 - - type: recall - value: 96.83794466403161 - task: - type: BitextMining - - dataset: - config: als_Latn-rus_Cyrl - name: MTEB FloresBitextMining (als_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.6907114624506 - - type: main_score - value: 98.6907114624506 - - type: precision - value: 98.6142480707698 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: bod_Tibt-rus_Cyrl - name: MTEB FloresBitextMining (bod_Tibt-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 1.0869565217391304 - - type: f1 - value: 0.9224649610442628 - - type: main_score - value: 0.9224649610442628 - - type: precision - value: 0.8894275740459898 - - type: recall - value: 1.0869565217391304 - task: - type: BitextMining - - dataset: - config: fij_Latn-rus_Cyrl - name: MTEB FloresBitextMining (fij_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 63.24110671936759 - - type: f1 - value: 60.373189068189525 - - type: main_score - value: 60.373189068189525 - - type: precision - value: 59.32326368115546 - - type: recall - value: 63.24110671936759 - task: - type: BitextMining - - dataset: - config: isl_Latn-rus_Cyrl - name: MTEB FloresBitextMining (isl_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.03162055335969 - - type: f1 - value: 87.3102634715907 - - type: main_score - value: 87.3102634715907 - - type: precision - value: 86.65991814698712 - - type: recall - value: 89.03162055335969 - task: - type: BitextMining - - dataset: - config: kon_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kon_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 73.91304347826086 - - type: f1 - value: 71.518235523573 - - type: main_score - value: 71.518235523573 - - type: precision - value: 70.58714102449801 - - type: recall - value: 73.91304347826086 - task: - type: BitextMining - - dataset: - config: mni_Beng-rus_Cyrl - name: MTEB FloresBitextMining (mni_Beng-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 29.545454545454547 - - type: f1 - value: 27.59513619889114 - - type: main_score - value: 27.59513619889114 - - type: precision - value: 26.983849851025344 - - type: recall - value: 29.545454545454547 - task: - type: BitextMining - - dataset: - config: ron_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ron_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.2094861660079 - - type: main_score - value: 99.2094861660079 - - type: precision - value: 99.1106719367589 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: szl_Latn-rus_Cyrl - name: MTEB FloresBitextMining (szl_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 86.26482213438736 - - type: f1 - value: 85.18912031587512 - - type: main_score - value: 85.18912031587512 - - type: precision - value: 84.77199409959775 - - type: recall - value: 86.26482213438736 - task: - type: BitextMining - - dataset: - config: vec_Latn-rus_Cyrl - name: MTEB FloresBitextMining (vec_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 85.67193675889328 - - type: f1 - value: 84.62529734716581 - - type: main_score - value: 84.62529734716581 - - type: precision - value: 84.2611422440705 - - type: recall - value: 85.67193675889328 - task: - type: BitextMining - - dataset: - config: amh_Ethi-rus_Cyrl - name: MTEB FloresBitextMining (amh_Ethi-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.76284584980237 - - type: f1 - value: 93.91735076517685 - - type: main_score - value: 93.91735076517685 - - type: precision - value: 93.57553798858147 - - type: recall - value: 94.76284584980237 - task: - type: BitextMining - - dataset: - config: bos_Latn-rus_Cyrl - name: MTEB FloresBitextMining (bos_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 99.05655938264634 - - type: main_score - value: 99.05655938264634 - - type: precision - value: 99.01185770750988 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: fin_Latn-rus_Cyrl - name: MTEB FloresBitextMining (fin_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.43741765480895 - - type: main_score - value: 97.43741765480895 - - type: precision - value: 97.1590909090909 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: ita_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ita_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.70355731225297 - - type: f1 - value: 99.60474308300395 - - type: main_score - value: 99.60474308300395 - - type: precision - value: 99.55533596837944 - - type: recall - value: 99.70355731225297 - task: - type: BitextMining - - dataset: - config: kor_Hang-rus_Cyrl - name: MTEB FloresBitextMining (kor_Hang-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.33201581027669 - - type: f1 - value: 96.49868247694334 - - type: main_score - value: 96.49868247694334 - - type: precision - value: 96.10507246376811 - - type: recall - value: 97.33201581027669 - task: - type: BitextMining - - dataset: - config: mos_Latn-rus_Cyrl - name: MTEB FloresBitextMining (mos_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 34.683794466403164 - - type: f1 - value: 32.766819308009076 - - type: main_score - value: 32.766819308009076 - - type: precision - value: 32.1637493670237 - - type: recall - value: 34.683794466403164 - task: - type: BitextMining - - dataset: - config: run_Latn-rus_Cyrl - name: MTEB FloresBitextMining (run_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 83.399209486166 - - type: f1 - value: 81.10578750604326 - - type: main_score - value: 81.10578750604326 - - type: precision - value: 80.16763162673529 - - type: recall - value: 83.399209486166 - task: - type: BitextMining - - dataset: - config: tam_Taml-rus_Cyrl - name: MTEB FloresBitextMining (tam_Taml-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.41897233201581 - - type: f1 - value: 98.01548089591567 - - type: main_score - value: 98.01548089591567 - - type: precision - value: 97.84020327498588 - - type: recall - value: 98.41897233201581 - task: - type: BitextMining - - dataset: - config: vie_Latn-rus_Cyrl - name: MTEB FloresBitextMining (vie_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.1106719367589 - - type: f1 - value: 98.81422924901186 - - type: main_score - value: 98.81422924901186 - - type: precision - value: 98.66600790513834 - - type: recall - value: 99.1106719367589 - task: - type: BitextMining - - dataset: - config: apc_Arab-rus_Cyrl - name: MTEB FloresBitextMining (apc_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.87351778656127 - - type: f1 - value: 92.10803689064558 - - type: main_score - value: 92.10803689064558 - - type: precision - value: 91.30434782608695 - - type: recall - value: 93.87351778656127 - task: - type: BitextMining - - dataset: - config: bug_Latn-rus_Cyrl - name: MTEB FloresBitextMining (bug_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 57.608695652173914 - - type: f1 - value: 54.95878654927162 - - type: main_score - value: 54.95878654927162 - - type: precision - value: 54.067987427805654 - - type: recall - value: 57.608695652173914 - task: - type: BitextMining - - dataset: - config: fon_Latn-rus_Cyrl - name: MTEB FloresBitextMining (fon_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 61.95652173913043 - - type: f1 - value: 58.06537275812945 - - type: main_score - value: 58.06537275812945 - - type: precision - value: 56.554057596959204 - - type: recall - value: 61.95652173913043 - task: - type: BitextMining - - dataset: - config: jav_Latn-rus_Cyrl - name: MTEB FloresBitextMining (jav_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.47826086956522 - - type: f1 - value: 92.4784405318002 - - type: main_score - value: 92.4784405318002 - - type: precision - value: 92.09168143201127 - - type: recall - value: 93.47826086956522 - task: - type: BitextMining - - dataset: - config: lao_Laoo-rus_Cyrl - name: MTEB FloresBitextMining (lao_Laoo-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.10671936758892 - - type: f1 - value: 89.76104922745239 - - type: main_score - value: 89.76104922745239 - - type: precision - value: 89.24754593232855 - - type: recall - value: 91.10671936758892 - task: - type: BitextMining - - dataset: - config: mri_Latn-rus_Cyrl - name: MTEB FloresBitextMining (mri_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 71.14624505928853 - - type: f1 - value: 68.26947125119062 - - type: main_score - value: 68.26947125119062 - - type: precision - value: 67.15942311051006 - - type: recall - value: 71.14624505928853 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ace_Arab - name: MTEB FloresBitextMining (rus_Cyrl-ace_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 19.565217391304348 - - type: f1 - value: 16.321465000323805 - - type: main_score - value: 16.321465000323805 - - type: precision - value: 15.478527409347508 - - type: recall - value: 19.565217391304348 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bam_Latn - name: MTEB FloresBitextMining (rus_Cyrl-bam_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 73.41897233201581 - - type: f1 - value: 68.77366228182746 - - type: main_score - value: 68.77366228182746 - - type: precision - value: 66.96012924273795 - - type: recall - value: 73.41897233201581 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-dzo_Tibt - name: MTEB FloresBitextMining (rus_Cyrl-dzo_Tibt) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 0.592885375494071 - - type: f1 - value: 0.02458062426370458 - - type: main_score - value: 0.02458062426370458 - - type: precision - value: 0.012824114724683876 - - type: recall - value: 0.592885375494071 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hin_Deva - name: MTEB FloresBitextMining (rus_Cyrl-hin_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.90118577075098 - - type: f1 - value: 99.86824769433464 - - type: main_score - value: 99.86824769433464 - - type: precision - value: 99.85177865612648 - - type: recall - value: 99.90118577075098 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-khm_Khmr - name: MTEB FloresBitextMining (rus_Cyrl-khm_Khmr) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.13438735177866 - - type: f1 - value: 96.24505928853755 - - type: main_score - value: 96.24505928853755 - - type: precision - value: 95.81686429512516 - - type: recall - value: 97.13438735177866 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mag_Deva - name: MTEB FloresBitextMining (rus_Cyrl-mag_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.50592885375494 - - type: f1 - value: 99.35770750988142 - - type: main_score - value: 99.35770750988142 - - type: precision - value: 99.29183135704875 - - type: recall - value: 99.50592885375494 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-pap_Latn - name: MTEB FloresBitextMining (rus_Cyrl-pap_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.93675889328063 - - type: f1 - value: 96.05072463768116 - - type: main_score - value: 96.05072463768116 - - type: precision - value: 95.66040843214758 - - type: recall - value: 96.93675889328063 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-sot_Latn - name: MTEB FloresBitextMining (rus_Cyrl-sot_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.67588932806325 - - type: f1 - value: 91.7786561264822 - - type: main_score - value: 91.7786561264822 - - type: precision - value: 90.91238471673255 - - type: recall - value: 93.67588932806325 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tur_Latn - name: MTEB FloresBitextMining (rus_Cyrl-tur_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.68247694334651 - - type: main_score - value: 98.68247694334651 - - type: precision - value: 98.51778656126481 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ace_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ace_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 74.1106719367589 - - type: f1 - value: 70.21737923911836 - - type: main_score - value: 70.21737923911836 - - type: precision - value: 68.7068791410511 - - type: recall - value: 74.1106719367589 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ban_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ban_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.7193675889328 - - type: f1 - value: 78.76470334510617 - - type: main_score - value: 78.76470334510617 - - type: precision - value: 77.76208475761422 - - type: recall - value: 81.7193675889328 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ell_Grek - name: MTEB FloresBitextMining (rus_Cyrl-ell_Grek) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.3201581027668 - - type: f1 - value: 97.76021080368908 - - type: main_score - value: 97.76021080368908 - - type: precision - value: 97.48023715415019 - - type: recall - value: 98.3201581027668 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hne_Deva - name: MTEB FloresBitextMining (rus_Cyrl-hne_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.51778656126481 - - type: f1 - value: 98.0566534914361 - - type: main_score - value: 98.0566534914361 - - type: precision - value: 97.82608695652173 - - type: recall - value: 98.51778656126481 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kik_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kik_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 80.73122529644269 - - type: f1 - value: 76.42689244220864 - - type: main_score - value: 76.42689244220864 - - type: precision - value: 74.63877909530083 - - type: recall - value: 80.73122529644269 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mai_Deva - name: MTEB FloresBitextMining (rus_Cyrl-mai_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.56719367588933 - - type: main_score - value: 98.56719367588933 - - type: precision - value: 98.40250329380763 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-pbt_Arab - name: MTEB FloresBitextMining (rus_Cyrl-pbt_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.5296442687747 - - type: f1 - value: 96.73913043478261 - - type: main_score - value: 96.73913043478261 - - type: precision - value: 96.36034255599473 - - type: recall - value: 97.5296442687747 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-spa_Latn - name: MTEB FloresBitextMining (rus_Cyrl-spa_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.20948616600789 - - type: main_score - value: 99.20948616600789 - - type: precision - value: 99.1106719367589 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-twi_Latn - name: MTEB FloresBitextMining (rus_Cyrl-twi_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 82.01581027667984 - - type: f1 - value: 78.064787822953 - - type: main_score - value: 78.064787822953 - - type: precision - value: 76.43272186750448 - - type: recall - value: 82.01581027667984 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-acm_Arab - name: MTEB FloresBitextMining (rus_Cyrl-acm_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.3201581027668 - - type: f1 - value: 97.76021080368908 - - type: main_score - value: 97.76021080368908 - - type: precision - value: 97.48023715415019 - - type: recall - value: 98.3201581027668 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bel_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-bel_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.22134387351778 - - type: f1 - value: 97.67786561264822 - - type: main_score - value: 97.67786561264822 - - type: precision - value: 97.4308300395257 - - type: recall - value: 98.22134387351778 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-eng_Latn - name: MTEB FloresBitextMining (rus_Cyrl-eng_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.70355731225297 - - type: f1 - value: 99.60474308300395 - - type: main_score - value: 99.60474308300395 - - type: precision - value: 99.55533596837944 - - type: recall - value: 99.70355731225297 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hrv_Latn - name: MTEB FloresBitextMining (rus_Cyrl-hrv_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.1106719367589 - - type: f1 - value: 98.83069828722002 - - type: main_score - value: 98.83069828722002 - - type: precision - value: 98.69894598155466 - - type: recall - value: 99.1106719367589 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kin_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kin_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.37944664031622 - - type: f1 - value: 91.53162055335969 - - type: main_score - value: 91.53162055335969 - - type: precision - value: 90.71475625823452 - - type: recall - value: 93.37944664031622 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mal_Mlym - name: MTEB FloresBitextMining (rus_Cyrl-mal_Mlym) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.07773386034255 - - type: main_score - value: 99.07773386034255 - - type: precision - value: 98.96245059288538 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-pes_Arab - name: MTEB FloresBitextMining (rus_Cyrl-pes_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.30368906455863 - - type: main_score - value: 98.30368906455863 - - type: precision - value: 98.10606060606061 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-srd_Latn - name: MTEB FloresBitextMining (rus_Cyrl-srd_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.03162055335969 - - type: f1 - value: 86.11048371917937 - - type: main_score - value: 86.11048371917937 - - type: precision - value: 84.86001317523056 - - type: recall - value: 89.03162055335969 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tzm_Tfng - name: MTEB FloresBitextMining (rus_Cyrl-tzm_Tfng) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 12.351778656126482 - - type: f1 - value: 10.112177999067715 - - type: main_score - value: 10.112177999067715 - - type: precision - value: 9.53495885438645 - - type: recall - value: 12.351778656126482 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-acq_Arab - name: MTEB FloresBitextMining (rus_Cyrl-acq_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.55072463768116 - - type: main_score - value: 98.55072463768116 - - type: precision - value: 98.36956521739131 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bem_Latn - name: MTEB FloresBitextMining (rus_Cyrl-bem_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 73.22134387351778 - - type: f1 - value: 68.30479412989295 - - type: main_score - value: 68.30479412989295 - - type: precision - value: 66.40073447632736 - - type: recall - value: 73.22134387351778 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-epo_Latn - name: MTEB FloresBitextMining (rus_Cyrl-epo_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.1106719367589 - - type: f1 - value: 98.81422924901186 - - type: main_score - value: 98.81422924901186 - - type: precision - value: 98.66600790513834 - - type: recall - value: 99.1106719367589 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hun_Latn - name: MTEB FloresBitextMining (rus_Cyrl-hun_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.83794466403161 - - type: f1 - value: 95.88274044795784 - - type: main_score - value: 95.88274044795784 - - type: precision - value: 95.45454545454545 - - type: recall - value: 96.83794466403161 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kir_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-kir_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.34387351778656 - - type: f1 - value: 95.49280429715212 - - type: main_score - value: 95.49280429715212 - - type: precision - value: 95.14163372859026 - - type: recall - value: 96.34387351778656 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mar_Deva - name: MTEB FloresBitextMining (rus_Cyrl-mar_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.28722002635047 - - type: main_score - value: 98.28722002635047 - - type: precision - value: 98.07312252964427 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-plt_Latn - name: MTEB FloresBitextMining (rus_Cyrl-plt_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 88.04347826086956 - - type: f1 - value: 85.14328063241106 - - type: main_score - value: 85.14328063241106 - - type: precision - value: 83.96339168078298 - - type: recall - value: 88.04347826086956 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-srp_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-srp_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.2094861660079 - - type: main_score - value: 99.2094861660079 - - type: precision - value: 99.1106719367589 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-uig_Arab - name: MTEB FloresBitextMining (rus_Cyrl-uig_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.19367588932806 - - type: f1 - value: 89.98541313758706 - - type: main_score - value: 89.98541313758706 - - type: precision - value: 89.01021080368906 - - type: recall - value: 92.19367588932806 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-aeb_Arab - name: MTEB FloresBitextMining (rus_Cyrl-aeb_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.8498023715415 - - type: f1 - value: 94.63109354413703 - - type: main_score - value: 94.63109354413703 - - type: precision - value: 94.05467720685111 - - type: recall - value: 95.8498023715415 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ben_Beng - name: MTEB FloresBitextMining (rus_Cyrl-ben_Beng) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.2094861660079 - - type: main_score - value: 99.2094861660079 - - type: precision - value: 99.1106719367589 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-est_Latn - name: MTEB FloresBitextMining (rus_Cyrl-est_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.55335968379447 - - type: f1 - value: 94.2588932806324 - - type: main_score - value: 94.2588932806324 - - type: precision - value: 93.65118577075098 - - type: recall - value: 95.55335968379447 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hye_Armn - name: MTEB FloresBitextMining (rus_Cyrl-hye_Armn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.28722002635045 - - type: main_score - value: 98.28722002635045 - - type: precision - value: 98.07312252964427 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kmb_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kmb_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 54.24901185770751 - - type: f1 - value: 49.46146674116913 - - type: main_score - value: 49.46146674116913 - - type: precision - value: 47.81033799314432 - - type: recall - value: 54.24901185770751 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-min_Arab - name: MTEB FloresBitextMining (rus_Cyrl-min_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 15.810276679841898 - - type: f1 - value: 13.271207641419332 - - type: main_score - value: 13.271207641419332 - - type: precision - value: 12.510673148766033 - - type: recall - value: 15.810276679841898 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-pol_Latn - name: MTEB FloresBitextMining (rus_Cyrl-pol_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.32674571805006 - - type: main_score - value: 98.32674571805006 - - type: precision - value: 98.14723320158103 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ssw_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ssw_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 80.8300395256917 - - type: f1 - value: 76.51717847370023 - - type: main_score - value: 76.51717847370023 - - type: precision - value: 74.74143610013175 - - type: recall - value: 80.8300395256917 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ukr_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-ukr_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.60474308300395 - - type: f1 - value: 99.4729907773386 - - type: main_score - value: 99.4729907773386 - - type: precision - value: 99.40711462450594 - - type: recall - value: 99.60474308300395 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-afr_Latn - name: MTEB FloresBitextMining (rus_Cyrl-afr_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.1106719367589 - - type: f1 - value: 98.81422924901186 - - type: main_score - value: 98.81422924901186 - - type: precision - value: 98.66600790513834 - - type: recall - value: 99.1106719367589 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bho_Deva - name: MTEB FloresBitextMining (rus_Cyrl-bho_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.6403162055336 - - type: f1 - value: 95.56982872200265 - - type: main_score - value: 95.56982872200265 - - type: precision - value: 95.0592885375494 - - type: recall - value: 96.6403162055336 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-eus_Latn - name: MTEB FloresBitextMining (rus_Cyrl-eus_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.62845849802372 - - type: f1 - value: 96.9038208168643 - - type: main_score - value: 96.9038208168643 - - type: precision - value: 96.55797101449275 - - type: recall - value: 97.62845849802372 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ibo_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ibo_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.2292490118577 - - type: f1 - value: 86.35234330886506 - - type: main_score - value: 86.35234330886506 - - type: precision - value: 85.09881422924902 - - type: recall - value: 89.2292490118577 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kmr_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kmr_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 83.49802371541502 - - type: f1 - value: 79.23630717108978 - - type: main_score - value: 79.23630717108978 - - type: precision - value: 77.48188405797102 - - type: recall - value: 83.49802371541502 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-min_Latn - name: MTEB FloresBitextMining (rus_Cyrl-min_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 79.34782608695652 - - type: f1 - value: 75.31689928429059 - - type: main_score - value: 75.31689928429059 - - type: precision - value: 73.91519410541149 - - type: recall - value: 79.34782608695652 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-por_Latn - name: MTEB FloresBitextMining (rus_Cyrl-por_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.54150197628458 - - type: f1 - value: 95.53218520609825 - - type: main_score - value: 95.53218520609825 - - type: precision - value: 95.07575757575756 - - type: recall - value: 96.54150197628458 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-sun_Latn - name: MTEB FloresBitextMining (rus_Cyrl-sun_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.2806324110672 - - type: f1 - value: 91.56973461321287 - - type: main_score - value: 91.56973461321287 - - type: precision - value: 90.84396334890405 - - type: recall - value: 93.2806324110672 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-umb_Latn - name: MTEB FloresBitextMining (rus_Cyrl-umb_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 51.87747035573123 - - type: f1 - value: 46.36591778884269 - - type: main_score - value: 46.36591778884269 - - type: precision - value: 44.57730391234227 - - type: recall - value: 51.87747035573123 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ajp_Arab - name: MTEB FloresBitextMining (rus_Cyrl-ajp_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.30368906455863 - - type: main_score - value: 98.30368906455863 - - type: precision - value: 98.10606060606061 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bjn_Arab - name: MTEB FloresBitextMining (rus_Cyrl-bjn_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 14.82213438735178 - - type: f1 - value: 12.365434276616856 - - type: main_score - value: 12.365434276616856 - - type: precision - value: 11.802079517180589 - - type: recall - value: 14.82213438735178 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ewe_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ewe_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 71.44268774703558 - - type: f1 - value: 66.74603174603175 - - type: main_score - value: 66.74603174603175 - - type: precision - value: 64.99933339607253 - - type: recall - value: 71.44268774703558 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ilo_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ilo_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 85.86956521739131 - - type: f1 - value: 83.00139015960917 - - type: main_score - value: 83.00139015960917 - - type: precision - value: 81.91411396574439 - - type: recall - value: 85.86956521739131 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-knc_Arab - name: MTEB FloresBitextMining (rus_Cyrl-knc_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 14.525691699604742 - - type: f1 - value: 12.618283715726806 - - type: main_score - value: 12.618283715726806 - - type: precision - value: 12.048458493742352 - - type: recall - value: 14.525691699604742 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mkd_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-mkd_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.22595520421606 - - type: main_score - value: 99.22595520421606 - - type: precision - value: 99.14361001317523 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-prs_Arab - name: MTEB FloresBitextMining (rus_Cyrl-prs_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.07773386034255 - - type: main_score - value: 99.07773386034255 - - type: precision - value: 98.96245059288538 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-swe_Latn - name: MTEB FloresBitextMining (rus_Cyrl-swe_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.07773386034256 - - type: main_score - value: 99.07773386034256 - - type: precision - value: 98.96245059288538 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-urd_Arab - name: MTEB FloresBitextMining (rus_Cyrl-urd_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.61660079051383 - - type: f1 - value: 98.15546772068511 - - type: main_score - value: 98.15546772068511 - - type: precision - value: 97.92490118577075 - - type: recall - value: 98.61660079051383 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-aka_Latn - name: MTEB FloresBitextMining (rus_Cyrl-aka_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.02766798418972 - - type: f1 - value: 76.73277809147375 - - type: main_score - value: 76.73277809147375 - - type: precision - value: 74.97404165882426 - - type: recall - value: 81.02766798418972 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bjn_Latn - name: MTEB FloresBitextMining (rus_Cyrl-bjn_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 86.7588932806324 - - type: f1 - value: 83.92064566965753 - - type: main_score - value: 83.92064566965753 - - type: precision - value: 82.83734079929732 - - type: recall - value: 86.7588932806324 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fao_Latn - name: MTEB FloresBitextMining (rus_Cyrl-fao_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 88.43873517786561 - - type: f1 - value: 85.48136645962732 - - type: main_score - value: 85.48136645962732 - - type: precision - value: 84.23418972332016 - - type: recall - value: 88.43873517786561 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ind_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ind_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.68247694334651 - - type: main_score - value: 98.68247694334651 - - type: precision - value: 98.51778656126481 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-knc_Latn - name: MTEB FloresBitextMining (rus_Cyrl-knc_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 45.8498023715415 - - type: f1 - value: 40.112030865489366 - - type: main_score - value: 40.112030865489366 - - type: precision - value: 38.28262440050776 - - type: recall - value: 45.8498023715415 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mlt_Latn - name: MTEB FloresBitextMining (rus_Cyrl-mlt_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.18181818181817 - - type: f1 - value: 91.30787690570298 - - type: main_score - value: 91.30787690570298 - - type: precision - value: 90.4983060417843 - - type: recall - value: 93.18181818181817 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-quy_Latn - name: MTEB FloresBitextMining (rus_Cyrl-quy_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 62.450592885375485 - - type: f1 - value: 57.28742975628178 - - type: main_score - value: 57.28742975628178 - - type: precision - value: 55.56854987623269 - - type: recall - value: 62.450592885375485 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-swh_Latn - name: MTEB FloresBitextMining (rus_Cyrl-swh_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.3201581027668 - - type: f1 - value: 97.77667984189723 - - type: main_score - value: 97.77667984189723 - - type: precision - value: 97.51317523056655 - - type: recall - value: 98.3201581027668 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-uzn_Latn - name: MTEB FloresBitextMining (rus_Cyrl-uzn_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.12252964426878 - - type: f1 - value: 97.59081498211933 - - type: main_score - value: 97.59081498211933 - - type: precision - value: 97.34848484848484 - - type: recall - value: 98.12252964426878 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-als_Latn - name: MTEB FloresBitextMining (rus_Cyrl-als_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.09420289855073 - - type: main_score - value: 99.09420289855073 - - type: precision - value: 98.99538866930172 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bod_Tibt - name: MTEB FloresBitextMining (rus_Cyrl-bod_Tibt) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 11.561264822134387 - - type: f1 - value: 8.121312045385636 - - type: main_score - value: 8.121312045385636 - - type: precision - value: 7.350577020893972 - - type: recall - value: 11.561264822134387 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fij_Latn - name: MTEB FloresBitextMining (rus_Cyrl-fij_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 72.23320158102767 - - type: f1 - value: 67.21000233846082 - - type: main_score - value: 67.21000233846082 - - type: precision - value: 65.3869439739005 - - type: recall - value: 72.23320158102767 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-isl_Latn - name: MTEB FloresBitextMining (rus_Cyrl-isl_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.99604743083005 - - type: f1 - value: 89.75955204216073 - - type: main_score - value: 89.75955204216073 - - type: precision - value: 88.7598814229249 - - type: recall - value: 91.99604743083005 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kon_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kon_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.81818181818183 - - type: f1 - value: 77.77800098452272 - - type: main_score - value: 77.77800098452272 - - type: precision - value: 76.1521268586486 - - type: recall - value: 81.81818181818183 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mni_Beng - name: MTEB FloresBitextMining (rus_Cyrl-mni_Beng) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 54.74308300395256 - - type: f1 - value: 48.97285299254615 - - type: main_score - value: 48.97285299254615 - - type: precision - value: 46.95125742968299 - - type: recall - value: 54.74308300395256 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ron_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ron_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.22134387351778 - - type: f1 - value: 97.64492753623189 - - type: main_score - value: 97.64492753623189 - - type: precision - value: 97.36495388669302 - - type: recall - value: 98.22134387351778 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-szl_Latn - name: MTEB FloresBitextMining (rus_Cyrl-szl_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.09486166007905 - - type: f1 - value: 90.10375494071147 - - type: main_score - value: 90.10375494071147 - - type: precision - value: 89.29606625258798 - - type: recall - value: 92.09486166007905 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-vec_Latn - name: MTEB FloresBitextMining (rus_Cyrl-vec_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.4901185770751 - - type: f1 - value: 90.51430453604365 - - type: main_score - value: 90.51430453604365 - - type: precision - value: 89.69367588932808 - - type: recall - value: 92.4901185770751 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-amh_Ethi - name: MTEB FloresBitextMining (rus_Cyrl-amh_Ethi) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.82608695652173 - - type: f1 - value: 97.11791831357048 - - type: main_score - value: 97.11791831357048 - - type: precision - value: 96.77206851119894 - - type: recall - value: 97.82608695652173 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bos_Latn - name: MTEB FloresBitextMining (rus_Cyrl-bos_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.55072463768116 - - type: main_score - value: 98.55072463768116 - - type: precision - value: 98.36956521739131 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fin_Latn - name: MTEB FloresBitextMining (rus_Cyrl-fin_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.65217391304348 - - type: f1 - value: 94.4235836627141 - - type: main_score - value: 94.4235836627141 - - type: precision - value: 93.84881422924902 - - type: recall - value: 95.65217391304348 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ita_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ita_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.55072463768117 - - type: main_score - value: 98.55072463768117 - - type: precision - value: 98.36956521739131 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kor_Hang - name: MTEB FloresBitextMining (rus_Cyrl-kor_Hang) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.55335968379447 - - type: f1 - value: 94.15349143610013 - - type: main_score - value: 94.15349143610013 - - type: precision - value: 93.49472990777339 - - type: recall - value: 95.55335968379447 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mos_Latn - name: MTEB FloresBitextMining (rus_Cyrl-mos_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 43.67588932806324 - - type: f1 - value: 38.84849721190082 - - type: main_score - value: 38.84849721190082 - - type: precision - value: 37.43294462099682 - - type: recall - value: 43.67588932806324 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-run_Latn - name: MTEB FloresBitextMining (rus_Cyrl-run_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 90.21739130434783 - - type: f1 - value: 87.37483530961792 - - type: main_score - value: 87.37483530961792 - - type: precision - value: 86.07872200263506 - - type: recall - value: 90.21739130434783 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tam_Taml - name: MTEB FloresBitextMining (rus_Cyrl-tam_Taml) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.2094861660079 - - type: main_score - value: 99.2094861660079 - - type: precision - value: 99.1106719367589 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-vie_Latn - name: MTEB FloresBitextMining (rus_Cyrl-vie_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.03557312252964 - - type: f1 - value: 96.13636363636364 - - type: main_score - value: 96.13636363636364 - - type: precision - value: 95.70981554677206 - - type: recall - value: 97.03557312252964 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-apc_Arab - name: MTEB FloresBitextMining (rus_Cyrl-apc_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.12252964426878 - - type: f1 - value: 97.49670619235836 - - type: main_score - value: 97.49670619235836 - - type: precision - value: 97.18379446640316 - - type: recall - value: 98.12252964426878 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bug_Latn - name: MTEB FloresBitextMining (rus_Cyrl-bug_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 67.29249011857708 - - type: f1 - value: 62.09268717667927 - - type: main_score - value: 62.09268717667927 - - type: precision - value: 60.28554009748714 - - type: recall - value: 67.29249011857708 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fon_Latn - name: MTEB FloresBitextMining (rus_Cyrl-fon_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 63.43873517786561 - - type: f1 - value: 57.66660107569199 - - type: main_score - value: 57.66660107569199 - - type: precision - value: 55.66676396919363 - - type: recall - value: 63.43873517786561 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-jav_Latn - name: MTEB FloresBitextMining (rus_Cyrl-jav_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.46640316205533 - - type: f1 - value: 92.89384528514964 - - type: main_score - value: 92.89384528514964 - - type: precision - value: 92.19367588932806 - - type: recall - value: 94.46640316205533 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lao_Laoo - name: MTEB FloresBitextMining (rus_Cyrl-lao_Laoo) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.23320158102767 - - type: f1 - value: 96.40974967061922 - - type: main_score - value: 96.40974967061922 - - type: precision - value: 96.034255599473 - - type: recall - value: 97.23320158102767 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mri_Latn - name: MTEB FloresBitextMining (rus_Cyrl-mri_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 76.77865612648222 - - type: f1 - value: 73.11286539547409 - - type: main_score - value: 73.11286539547409 - - type: precision - value: 71.78177214337046 - - type: recall - value: 76.77865612648222 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-taq_Latn - name: MTEB FloresBitextMining (rus_Cyrl-taq_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 41.99604743083004 - - type: f1 - value: 37.25127063318763 - - type: main_score - value: 37.25127063318763 - - type: precision - value: 35.718929186985726 - - type: recall - value: 41.99604743083004 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-war_Latn - name: MTEB FloresBitextMining (rus_Cyrl-war_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.55335968379447 - - type: f1 - value: 94.1699604743083 - - type: main_score - value: 94.1699604743083 - - type: precision - value: 93.52766798418972 - - type: recall - value: 95.55335968379447 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-arb_Arab - name: MTEB FloresBitextMining (rus_Cyrl-arb_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.60474308300395 - - type: f1 - value: 99.4729907773386 - - type: main_score - value: 99.4729907773386 - - type: precision - value: 99.40711462450594 - - type: recall - value: 99.60474308300395 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bul_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-bul_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.70355731225297 - - type: f1 - value: 99.60474308300395 - - type: main_score - value: 99.60474308300395 - - type: precision - value: 99.55533596837944 - - type: recall - value: 99.70355731225297 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fra_Latn - name: MTEB FloresBitextMining (rus_Cyrl-fra_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.60474308300395 - - type: f1 - value: 99.47299077733861 - - type: main_score - value: 99.47299077733861 - - type: precision - value: 99.40711462450594 - - type: recall - value: 99.60474308300395 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-jpn_Jpan - name: MTEB FloresBitextMining (rus_Cyrl-jpn_Jpan) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.44268774703558 - - type: f1 - value: 95.30632411067194 - - type: main_score - value: 95.30632411067194 - - type: precision - value: 94.76284584980237 - - type: recall - value: 96.44268774703558 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lij_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lij_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 90.21739130434783 - - type: f1 - value: 87.4703557312253 - - type: main_score - value: 87.4703557312253 - - type: precision - value: 86.29611330698287 - - type: recall - value: 90.21739130434783 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mya_Mymr - name: MTEB FloresBitextMining (rus_Cyrl-mya_Mymr) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.364953886693 - - type: main_score - value: 97.364953886693 - - type: precision - value: 97.03557312252964 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-sag_Latn - name: MTEB FloresBitextMining (rus_Cyrl-sag_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 54.841897233201585 - - type: f1 - value: 49.61882037503349 - - type: main_score - value: 49.61882037503349 - - type: precision - value: 47.831968755881796 - - type: recall - value: 54.841897233201585 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-taq_Tfng - name: MTEB FloresBitextMining (rus_Cyrl-taq_Tfng) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 15.316205533596838 - - type: f1 - value: 11.614836360389717 - - type: main_score - value: 11.614836360389717 - - type: precision - value: 10.741446193235223 - - type: recall - value: 15.316205533596838 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-wol_Latn - name: MTEB FloresBitextMining (rus_Cyrl-wol_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 67.88537549407114 - - type: f1 - value: 62.2536417249856 - - type: main_score - value: 62.2536417249856 - - type: precision - value: 60.27629128666678 - - type: recall - value: 67.88537549407114 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-arb_Latn - name: MTEB FloresBitextMining (rus_Cyrl-arb_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 27.766798418972332 - - type: f1 - value: 23.39674889624077 - - type: main_score - value: 23.39674889624077 - - type: precision - value: 22.28521155585345 - - type: recall - value: 27.766798418972332 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-cat_Latn - name: MTEB FloresBitextMining (rus_Cyrl-cat_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.23320158102767 - - type: f1 - value: 96.42151326933936 - - type: main_score - value: 96.42151326933936 - - type: precision - value: 96.04743083003953 - - type: recall - value: 97.23320158102767 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fur_Latn - name: MTEB FloresBitextMining (rus_Cyrl-fur_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 88.63636363636364 - - type: f1 - value: 85.80792396009788 - - type: main_score - value: 85.80792396009788 - - type: precision - value: 84.61508901726293 - - type: recall - value: 88.63636363636364 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kab_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kab_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 48.12252964426877 - - type: f1 - value: 43.05387582971066 - - type: main_score - value: 43.05387582971066 - - type: precision - value: 41.44165117538212 - - type: recall - value: 48.12252964426877 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lim_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lim_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.81818181818183 - - type: f1 - value: 77.81676163099087 - - type: main_score - value: 77.81676163099087 - - type: precision - value: 76.19565217391305 - - type: recall - value: 81.81818181818183 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-nld_Latn - name: MTEB FloresBitextMining (rus_Cyrl-nld_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.33201581027669 - - type: f1 - value: 96.4756258234519 - - type: main_score - value: 96.4756258234519 - - type: precision - value: 96.06389986824769 - - type: recall - value: 97.33201581027669 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-san_Deva - name: MTEB FloresBitextMining (rus_Cyrl-san_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.47826086956522 - - type: f1 - value: 91.70289855072463 - - type: main_score - value: 91.70289855072463 - - type: precision - value: 90.9370882740448 - - type: recall - value: 93.47826086956522 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tat_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-tat_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.72727272727273 - - type: f1 - value: 97.00263504611331 - - type: main_score - value: 97.00263504611331 - - type: precision - value: 96.65678524374177 - - type: recall - value: 97.72727272727273 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-xho_Latn - name: MTEB FloresBitextMining (rus_Cyrl-xho_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.08300395256917 - - type: f1 - value: 91.12977602108036 - - type: main_score - value: 91.12977602108036 - - type: precision - value: 90.22562582345192 - - type: recall - value: 93.08300395256917 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ars_Arab - name: MTEB FloresBitextMining (rus_Cyrl-ars_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.40711462450594 - - type: f1 - value: 99.2094861660079 - - type: main_score - value: 99.2094861660079 - - type: precision - value: 99.1106719367589 - - type: recall - value: 99.40711462450594 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ceb_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ceb_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.65217391304348 - - type: f1 - value: 94.3544137022398 - - type: main_score - value: 94.3544137022398 - - type: precision - value: 93.76646903820817 - - type: recall - value: 95.65217391304348 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fuv_Latn - name: MTEB FloresBitextMining (rus_Cyrl-fuv_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 51.18577075098815 - - type: f1 - value: 44.5990252610806 - - type: main_score - value: 44.5990252610806 - - type: precision - value: 42.34331599450177 - - type: recall - value: 51.18577075098815 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kac_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kac_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 46.93675889328063 - - type: f1 - value: 41.79004018701787 - - type: main_score - value: 41.79004018701787 - - type: precision - value: 40.243355662392624 - - type: recall - value: 46.93675889328063 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lin_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lin_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.50197628458498 - - type: f1 - value: 89.1205533596838 - - type: main_score - value: 89.1205533596838 - - type: precision - value: 88.07147562582345 - - type: recall - value: 91.50197628458498 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-nno_Latn - name: MTEB FloresBitextMining (rus_Cyrl-nno_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.81422924901186 - - type: f1 - value: 98.41897233201581 - - type: main_score - value: 98.41897233201581 - - type: precision - value: 98.22134387351778 - - type: recall - value: 98.81422924901186 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-sat_Olck - name: MTEB FloresBitextMining (rus_Cyrl-sat_Olck) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 2.371541501976284 - - type: f1 - value: 1.0726274943087382 - - type: main_score - value: 1.0726274943087382 - - type: precision - value: 0.875279634748803 - - type: recall - value: 2.371541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tel_Telu - name: MTEB FloresBitextMining (rus_Cyrl-tel_Telu) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.68247694334651 - - type: main_score - value: 98.68247694334651 - - type: precision - value: 98.51778656126481 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ydd_Hebr - name: MTEB FloresBitextMining (rus_Cyrl-ydd_Hebr) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.42687747035573 - - type: f1 - value: 86.47609636740073 - - type: main_score - value: 86.47609636740073 - - type: precision - value: 85.13669301712781 - - type: recall - value: 89.42687747035573 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ary_Arab - name: MTEB FloresBitextMining (rus_Cyrl-ary_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.82213438735178 - - type: f1 - value: 87.04545454545456 - - type: main_score - value: 87.04545454545456 - - type: precision - value: 85.76910408432148 - - type: recall - value: 89.82213438735178 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ces_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ces_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.9459815546772 - - type: main_score - value: 98.9459815546772 - - type: precision - value: 98.81422924901186 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-gaz_Latn - name: MTEB FloresBitextMining (rus_Cyrl-gaz_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 64.9209486166008 - - type: f1 - value: 58.697458119394874 - - type: main_score - value: 58.697458119394874 - - type: precision - value: 56.43402189597842 - - type: recall - value: 64.9209486166008 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kam_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kam_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 59.18972332015811 - - type: f1 - value: 53.19031511966295 - - type: main_score - value: 53.19031511966295 - - type: precision - value: 51.08128357343655 - - type: recall - value: 59.18972332015811 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lit_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lit_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.54150197628458 - - type: f1 - value: 95.5368906455863 - - type: main_score - value: 95.5368906455863 - - type: precision - value: 95.0592885375494 - - type: recall - value: 96.54150197628458 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-nob_Latn - name: MTEB FloresBitextMining (rus_Cyrl-nob_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.12252964426878 - - type: f1 - value: 97.51317523056655 - - type: main_score - value: 97.51317523056655 - - type: precision - value: 97.2167325428195 - - type: recall - value: 98.12252964426878 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-scn_Latn - name: MTEB FloresBitextMining (rus_Cyrl-scn_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 84.0909090909091 - - type: f1 - value: 80.37000439174352 - - type: main_score - value: 80.37000439174352 - - type: precision - value: 78.83994628559846 - - type: recall - value: 84.0909090909091 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tgk_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-tgk_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.68774703557312 - - type: f1 - value: 90.86344814605684 - - type: main_score - value: 90.86344814605684 - - type: precision - value: 90.12516469038208 - - type: recall - value: 92.68774703557312 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-yor_Latn - name: MTEB FloresBitextMining (rus_Cyrl-yor_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 72.13438735177866 - - type: f1 - value: 66.78759646150951 - - type: main_score - value: 66.78759646150951 - - type: precision - value: 64.85080192096002 - - type: recall - value: 72.13438735177866 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-arz_Arab - name: MTEB FloresBitextMining (rus_Cyrl-arz_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.364953886693 - - type: main_score - value: 97.364953886693 - - type: precision - value: 97.03557312252964 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-cjk_Latn - name: MTEB FloresBitextMining (rus_Cyrl-cjk_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 51.976284584980235 - - type: f1 - value: 46.468762353149714 - - type: main_score - value: 46.468762353149714 - - type: precision - value: 44.64073366247278 - - type: recall - value: 51.976284584980235 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-gla_Latn - name: MTEB FloresBitextMining (rus_Cyrl-gla_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 79.74308300395256 - - type: f1 - value: 75.55611165294958 - - type: main_score - value: 75.55611165294958 - - type: precision - value: 73.95033408620365 - - type: recall - value: 79.74308300395256 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kan_Knda - name: MTEB FloresBitextMining (rus_Cyrl-kan_Knda) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.96245059288538 - - type: main_score - value: 98.96245059288538 - - type: precision - value: 98.84716732542819 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lmo_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lmo_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 82.41106719367589 - - type: f1 - value: 78.56413514022209 - - type: main_score - value: 78.56413514022209 - - type: precision - value: 77.15313068573938 - - type: recall - value: 82.41106719367589 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-npi_Deva - name: MTEB FloresBitextMining (rus_Cyrl-npi_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.3201581027668 - - type: main_score - value: 98.3201581027668 - - type: precision - value: 98.12252964426878 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-shn_Mymr - name: MTEB FloresBitextMining (rus_Cyrl-shn_Mymr) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 57.11462450592886 - - type: f1 - value: 51.51361369197337 - - type: main_score - value: 51.51361369197337 - - type: precision - value: 49.71860043649573 - - type: recall - value: 57.11462450592886 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tgl_Latn - name: MTEB FloresBitextMining (rus_Cyrl-tgl_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.82608695652173 - - type: f1 - value: 97.18379446640316 - - type: main_score - value: 97.18379446640316 - - type: precision - value: 96.88735177865613 - - type: recall - value: 97.82608695652173 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-yue_Hant - name: MTEB FloresBitextMining (rus_Cyrl-yue_Hant) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.09420289855072 - - type: main_score - value: 99.09420289855072 - - type: precision - value: 98.9953886693017 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-asm_Beng - name: MTEB FloresBitextMining (rus_Cyrl-asm_Beng) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.55335968379447 - - type: f1 - value: 94.16007905138339 - - type: main_score - value: 94.16007905138339 - - type: precision - value: 93.50296442687747 - - type: recall - value: 95.55335968379447 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ckb_Arab - name: MTEB FloresBitextMining (rus_Cyrl-ckb_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.88537549407114 - - type: f1 - value: 90.76745718050066 - - type: main_score - value: 90.76745718050066 - - type: precision - value: 89.80072463768116 - - type: recall - value: 92.88537549407114 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-gle_Latn - name: MTEB FloresBitextMining (rus_Cyrl-gle_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.699604743083 - - type: f1 - value: 89.40899680030115 - - type: main_score - value: 89.40899680030115 - - type: precision - value: 88.40085638998683 - - type: recall - value: 91.699604743083 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kas_Arab - name: MTEB FloresBitextMining (rus_Cyrl-kas_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 88.3399209486166 - - type: f1 - value: 85.14351590438548 - - type: main_score - value: 85.14351590438548 - - type: precision - value: 83.72364953886692 - - type: recall - value: 88.3399209486166 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ltg_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ltg_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 83.399209486166 - - type: f1 - value: 79.88408934061107 - - type: main_score - value: 79.88408934061107 - - type: precision - value: 78.53794509179885 - - type: recall - value: 83.399209486166 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-nso_Latn - name: MTEB FloresBitextMining (rus_Cyrl-nso_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.20553359683794 - - type: f1 - value: 88.95406635525212 - - type: main_score - value: 88.95406635525212 - - type: precision - value: 88.01548089591567 - - type: recall - value: 91.20553359683794 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-sin_Sinh - name: MTEB FloresBitextMining (rus_Cyrl-sin_Sinh) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.56719367588933 - - type: main_score - value: 98.56719367588933 - - type: precision - value: 98.40250329380763 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tha_Thai - name: MTEB FloresBitextMining (rus_Cyrl-tha_Thai) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.94861660079052 - - type: f1 - value: 94.66403162055336 - - type: main_score - value: 94.66403162055336 - - type: precision - value: 94.03820816864295 - - type: recall - value: 95.94861660079052 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-zho_Hans - name: MTEB FloresBitextMining (rus_Cyrl-zho_Hans) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.4308300395257 - - type: f1 - value: 96.5909090909091 - - type: main_score - value: 96.5909090909091 - - type: precision - value: 96.17918313570487 - - type: recall - value: 97.4308300395257 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ast_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ast_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.46640316205533 - - type: f1 - value: 92.86890645586297 - - type: main_score - value: 92.86890645586297 - - type: precision - value: 92.14756258234519 - - type: recall - value: 94.46640316205533 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-crh_Latn - name: MTEB FloresBitextMining (rus_Cyrl-crh_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.66403162055336 - - type: f1 - value: 93.2663592446201 - - type: main_score - value: 93.2663592446201 - - type: precision - value: 92.66716073781292 - - type: recall - value: 94.66403162055336 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-glg_Latn - name: MTEB FloresBitextMining (rus_Cyrl-glg_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.81422924901186 - - type: f1 - value: 98.46837944664031 - - type: main_score - value: 98.46837944664031 - - type: precision - value: 98.3201581027668 - - type: recall - value: 98.81422924901186 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kas_Deva - name: MTEB FloresBitextMining (rus_Cyrl-kas_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 69.1699604743083 - - type: f1 - value: 63.05505292906477 - - type: main_score - value: 63.05505292906477 - - type: precision - value: 60.62594108789761 - - type: recall - value: 69.1699604743083 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ltz_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ltz_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.40316205533597 - - type: f1 - value: 89.26571616789009 - - type: main_score - value: 89.26571616789009 - - type: precision - value: 88.40179747788443 - - type: recall - value: 91.40316205533597 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-nus_Latn - name: MTEB FloresBitextMining (rus_Cyrl-nus_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 38.93280632411067 - - type: f1 - value: 33.98513032905371 - - type: main_score - value: 33.98513032905371 - - type: precision - value: 32.56257884802308 - - type: recall - value: 38.93280632411067 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-slk_Latn - name: MTEB FloresBitextMining (rus_Cyrl-slk_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.42094861660078 - - type: main_score - value: 97.42094861660078 - - type: precision - value: 97.14262187088273 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tir_Ethi - name: MTEB FloresBitextMining (rus_Cyrl-tir_Ethi) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.30434782608695 - - type: f1 - value: 88.78129117259552 - - type: main_score - value: 88.78129117259552 - - type: precision - value: 87.61528326745717 - - type: recall - value: 91.30434782608695 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-zho_Hant - name: MTEB FloresBitextMining (rus_Cyrl-zho_Hant) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.1106719367589 - - type: f1 - value: 98.81422924901186 - - type: main_score - value: 98.81422924901186 - - type: precision - value: 98.66600790513834 - - type: recall - value: 99.1106719367589 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-awa_Deva - name: MTEB FloresBitextMining (rus_Cyrl-awa_Deva) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.12252964426878 - - type: f1 - value: 97.70092226613966 - - type: main_score - value: 97.70092226613966 - - type: precision - value: 97.50494071146245 - - type: recall - value: 98.12252964426878 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-cym_Latn - name: MTEB FloresBitextMining (rus_Cyrl-cym_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.94861660079052 - - type: f1 - value: 94.74308300395256 - - type: main_score - value: 94.74308300395256 - - type: precision - value: 94.20289855072464 - - type: recall - value: 95.94861660079052 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-grn_Latn - name: MTEB FloresBitextMining (rus_Cyrl-grn_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 77.96442687747036 - - type: f1 - value: 73.64286789187975 - - type: main_score - value: 73.64286789187975 - - type: precision - value: 71.99324893260821 - - type: recall - value: 77.96442687747036 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kat_Geor - name: MTEB FloresBitextMining (rus_Cyrl-kat_Geor) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.56719367588933 - - type: main_score - value: 98.56719367588933 - - type: precision - value: 98.40250329380764 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lua_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lua_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 72.03557312252964 - - type: f1 - value: 67.23928163404449 - - type: main_score - value: 67.23928163404449 - - type: precision - value: 65.30797101449275 - - type: recall - value: 72.03557312252964 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-nya_Latn - name: MTEB FloresBitextMining (rus_Cyrl-nya_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.29249011857708 - - type: f1 - value: 90.0494071146245 - - type: main_score - value: 90.0494071146245 - - type: precision - value: 89.04808959156786 - - type: recall - value: 92.29249011857708 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-slv_Latn - name: MTEB FloresBitextMining (rus_Cyrl-slv_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.30368906455863 - - type: main_score - value: 98.30368906455863 - - type: precision - value: 98.10606060606061 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tpi_Latn - name: MTEB FloresBitextMining (rus_Cyrl-tpi_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 80.53359683794467 - - type: f1 - value: 76.59481822525301 - - type: main_score - value: 76.59481822525301 - - type: precision - value: 75.12913223140497 - - type: recall - value: 80.53359683794467 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-zsm_Latn - name: MTEB FloresBitextMining (rus_Cyrl-zsm_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.33201581027669 - - type: f1 - value: 96.58620365142104 - - type: main_score - value: 96.58620365142104 - - type: precision - value: 96.26152832674572 - - type: recall - value: 97.33201581027669 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ayr_Latn - name: MTEB FloresBitextMining (rus_Cyrl-ayr_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 45.55335968379446 - - type: f1 - value: 40.13076578531388 - - type: main_score - value: 40.13076578531388 - - type: precision - value: 38.398064362362355 - - type: recall - value: 45.55335968379446 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-dan_Latn - name: MTEB FloresBitextMining (rus_Cyrl-dan_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.68247694334651 - - type: main_score - value: 98.68247694334651 - - type: precision - value: 98.51778656126481 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-guj_Gujr - name: MTEB FloresBitextMining (rus_Cyrl-guj_Gujr) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.68247694334651 - - type: main_score - value: 98.68247694334651 - - type: precision - value: 98.51778656126481 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kaz_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-kaz_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.81422924901186 - - type: f1 - value: 98.43544137022398 - - type: main_score - value: 98.43544137022398 - - type: precision - value: 98.25428194993412 - - type: recall - value: 98.81422924901186 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lug_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lug_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 82.21343873517787 - - type: f1 - value: 77.97485726833554 - - type: main_score - value: 77.97485726833554 - - type: precision - value: 76.22376717485415 - - type: recall - value: 82.21343873517787 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-oci_Latn - name: MTEB FloresBitextMining (rus_Cyrl-oci_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.87351778656127 - - type: f1 - value: 92.25319969885187 - - type: main_score - value: 92.25319969885187 - - type: precision - value: 91.5638528138528 - - type: recall - value: 93.87351778656127 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-smo_Latn - name: MTEB FloresBitextMining (rus_Cyrl-smo_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 84.88142292490119 - - type: f1 - value: 81.24364765669114 - - type: main_score - value: 81.24364765669114 - - type: precision - value: 79.69991416137661 - - type: recall - value: 84.88142292490119 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tsn_Latn - name: MTEB FloresBitextMining (rus_Cyrl-tsn_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 87.05533596837944 - - type: f1 - value: 83.90645586297761 - - type: main_score - value: 83.90645586297761 - - type: precision - value: 82.56752305665349 - - type: recall - value: 87.05533596837944 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-zul_Latn - name: MTEB FloresBitextMining (rus_Cyrl-zul_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.15810276679841 - - type: f1 - value: 93.77140974967062 - - type: main_score - value: 93.77140974967062 - - type: precision - value: 93.16534914361002 - - type: recall - value: 95.15810276679841 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-azb_Arab - name: MTEB FloresBitextMining (rus_Cyrl-azb_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.91699604743083 - - type: f1 - value: 77.18050065876152 - - type: main_score - value: 77.18050065876152 - - type: precision - value: 75.21519543258673 - - type: recall - value: 81.91699604743083 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-deu_Latn - name: MTEB FloresBitextMining (rus_Cyrl-deu_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.50592885375494 - - type: f1 - value: 99.34123847167325 - - type: main_score - value: 99.34123847167325 - - type: precision - value: 99.2588932806324 - - type: recall - value: 99.50592885375494 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hat_Latn - name: MTEB FloresBitextMining (rus_Cyrl-hat_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.00790513833992 - - type: f1 - value: 88.69126043039086 - - type: main_score - value: 88.69126043039086 - - type: precision - value: 87.75774044795784 - - type: recall - value: 91.00790513833992 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kbp_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kbp_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 47.233201581027664 - - type: f1 - value: 43.01118618096943 - - type: main_score - value: 43.01118618096943 - - type: precision - value: 41.739069205043556 - - type: recall - value: 47.233201581027664 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-luo_Latn - name: MTEB FloresBitextMining (rus_Cyrl-luo_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 60.47430830039525 - - type: f1 - value: 54.83210565429816 - - type: main_score - value: 54.83210565429816 - - type: precision - value: 52.81630744284779 - - type: recall - value: 60.47430830039525 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ory_Orya - name: MTEB FloresBitextMining (rus_Cyrl-ory_Orya) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.1106719367589 - - type: f1 - value: 98.83069828722003 - - type: main_score - value: 98.83069828722003 - - type: precision - value: 98.69894598155467 - - type: recall - value: 99.1106719367589 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-sna_Latn - name: MTEB FloresBitextMining (rus_Cyrl-sna_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.72332015810277 - - type: f1 - value: 87.30013645774514 - - type: main_score - value: 87.30013645774514 - - type: precision - value: 86.25329380764163 - - type: recall - value: 89.72332015810277 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tso_Latn - name: MTEB FloresBitextMining (rus_Cyrl-tso_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 84.38735177865613 - - type: f1 - value: 80.70424744337788 - - type: main_score - value: 80.70424744337788 - - type: precision - value: 79.18560606060606 - - type: recall - value: 84.38735177865613 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-azj_Latn - name: MTEB FloresBitextMining (rus_Cyrl-azj_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.33201581027669 - - type: f1 - value: 96.56455862977602 - - type: main_score - value: 96.56455862977602 - - type: precision - value: 96.23682476943345 - - type: recall - value: 97.33201581027669 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-dik_Latn - name: MTEB FloresBitextMining (rus_Cyrl-dik_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 46.047430830039524 - - type: f1 - value: 40.05513069495283 - - type: main_score - value: 40.05513069495283 - - type: precision - value: 38.072590197096126 - - type: recall - value: 46.047430830039524 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hau_Latn - name: MTEB FloresBitextMining (rus_Cyrl-hau_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 87.94466403162056 - - type: f1 - value: 84.76943346508563 - - type: main_score - value: 84.76943346508563 - - type: precision - value: 83.34486166007905 - - type: recall - value: 87.94466403162056 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kea_Latn - name: MTEB FloresBitextMining (rus_Cyrl-kea_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.42687747035573 - - type: f1 - value: 86.83803021747684 - - type: main_score - value: 86.83803021747684 - - type: precision - value: 85.78416149068323 - - type: recall - value: 89.42687747035573 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lus_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lus_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 68.97233201581028 - - type: f1 - value: 64.05480726292745 - - type: main_score - value: 64.05480726292745 - - type: precision - value: 62.42670749487858 - - type: recall - value: 68.97233201581028 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-pag_Latn - name: MTEB FloresBitextMining (rus_Cyrl-pag_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 78.75494071146245 - - type: f1 - value: 74.58573558401933 - - type: main_score - value: 74.58573558401933 - - type: precision - value: 73.05532028358115 - - type: recall - value: 78.75494071146245 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-snd_Arab - name: MTEB FloresBitextMining (rus_Cyrl-snd_Arab) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.8498023715415 - - type: f1 - value: 94.56521739130434 - - type: main_score - value: 94.56521739130434 - - type: precision - value: 93.97233201581028 - - type: recall - value: 95.8498023715415 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tuk_Latn - name: MTEB FloresBitextMining (rus_Cyrl-tuk_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 68.08300395256917 - - type: f1 - value: 62.93565240205557 - - type: main_score - value: 62.93565240205557 - - type: precision - value: 61.191590257043934 - - type: recall - value: 68.08300395256917 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bak_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-bak_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.04743083003953 - - type: f1 - value: 94.86824769433464 - - type: main_score - value: 94.86824769433464 - - type: precision - value: 94.34288537549406 - - type: recall - value: 96.04743083003953 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-dyu_Latn - name: MTEB FloresBitextMining (rus_Cyrl-dyu_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 37.45059288537549 - - type: f1 - value: 31.670482312800807 - - type: main_score - value: 31.670482312800807 - - type: precision - value: 29.99928568357422 - - type: recall - value: 37.45059288537549 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-heb_Hebr - name: MTEB FloresBitextMining (rus_Cyrl-heb_Hebr) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.23320158102767 - - type: f1 - value: 96.38998682476942 - - type: main_score - value: 96.38998682476942 - - type: precision - value: 95.99802371541502 - - type: recall - value: 97.23320158102767 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-khk_Cyrl - name: MTEB FloresBitextMining (rus_Cyrl-khk_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.41897233201581 - - type: f1 - value: 98.00724637681158 - - type: main_score - value: 98.00724637681158 - - type: precision - value: 97.82938076416336 - - type: recall - value: 98.41897233201581 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lvs_Latn - name: MTEB FloresBitextMining (rus_Cyrl-lvs_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.4308300395257 - - type: f1 - value: 96.61396574440053 - - type: main_score - value: 96.61396574440053 - - type: precision - value: 96.2203557312253 - - type: recall - value: 97.4308300395257 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-pan_Guru - name: MTEB FloresBitextMining (rus_Cyrl-pan_Guru) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.07773386034256 - - type: main_score - value: 99.07773386034256 - - type: precision - value: 98.96245059288538 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-som_Latn - name: MTEB FloresBitextMining (rus_Cyrl-som_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 87.74703557312253 - - type: f1 - value: 84.52898550724638 - - type: main_score - value: 84.52898550724638 - - type: precision - value: 83.09288537549409 - - type: recall - value: 87.74703557312253 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tum_Latn - name: MTEB FloresBitextMining (rus_Cyrl-tum_Latn) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 87.15415019762845 - - type: f1 - value: 83.85069640504425 - - type: main_score - value: 83.85069640504425 - - type: precision - value: 82.43671183888576 - - type: recall - value: 87.15415019762845 - task: - type: BitextMining - - dataset: - config: taq_Latn-rus_Cyrl - name: MTEB FloresBitextMining (taq_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 28.55731225296443 - - type: f1 - value: 26.810726360049568 - - type: main_score - value: 26.810726360049568 - - type: precision - value: 26.260342858265577 - - type: recall - value: 28.55731225296443 - task: - type: BitextMining - - dataset: - config: war_Latn-rus_Cyrl - name: MTEB FloresBitextMining (war_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.86166007905138 - - type: f1 - value: 94.03147083483051 - - type: main_score - value: 94.03147083483051 - - type: precision - value: 93.70653606003322 - - type: recall - value: 94.86166007905138 - task: - type: BitextMining - - dataset: - config: arb_Arab-rus_Cyrl - name: MTEB FloresBitextMining (arb_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.34387351778656 - - type: f1 - value: 95.23056653491436 - - type: main_score - value: 95.23056653491436 - - type: precision - value: 94.70520421607378 - - type: recall - value: 96.34387351778656 - task: - type: BitextMining - - dataset: - config: bul_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (bul_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.90118577075098 - - type: f1 - value: 99.86824769433464 - - type: main_score - value: 99.86824769433464 - - type: precision - value: 99.85177865612648 - - type: recall - value: 99.90118577075098 - task: - type: BitextMining - - dataset: - config: fra_Latn-rus_Cyrl - name: MTEB FloresBitextMining (fra_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.9459815546772 - - type: main_score - value: 98.9459815546772 - - type: precision - value: 98.81422924901186 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: jpn_Jpan-rus_Cyrl - name: MTEB FloresBitextMining (jpn_Jpan-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.3201581027668 - - type: f1 - value: 97.76021080368905 - - type: main_score - value: 97.76021080368905 - - type: precision - value: 97.48023715415019 - - type: recall - value: 98.3201581027668 - task: - type: BitextMining - - dataset: - config: lij_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lij_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 83.49802371541502 - - type: f1 - value: 81.64800059239636 - - type: main_score - value: 81.64800059239636 - - type: precision - value: 80.9443055878478 - - type: recall - value: 83.49802371541502 - task: - type: BitextMining - - dataset: - config: mya_Mymr-rus_Cyrl - name: MTEB FloresBitextMining (mya_Mymr-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 90.21739130434783 - - type: f1 - value: 88.76776366313682 - - type: main_score - value: 88.76776366313682 - - type: precision - value: 88.18370446119435 - - type: recall - value: 90.21739130434783 - task: - type: BitextMining - - dataset: - config: sag_Latn-rus_Cyrl - name: MTEB FloresBitextMining (sag_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 41.699604743083 - - type: f1 - value: 39.53066322643847 - - type: main_score - value: 39.53066322643847 - - type: precision - value: 38.822876239229274 - - type: recall - value: 41.699604743083 - task: - type: BitextMining - - dataset: - config: taq_Tfng-rus_Cyrl - name: MTEB FloresBitextMining (taq_Tfng-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 10.67193675889328 - - type: f1 - value: 9.205744965817951 - - type: main_score - value: 9.205744965817951 - - type: precision - value: 8.85195219073817 - - type: recall - value: 10.67193675889328 - task: - type: BitextMining - - dataset: - config: wol_Latn-rus_Cyrl - name: MTEB FloresBitextMining (wol_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 63.537549407114625 - - type: f1 - value: 60.65190727391827 - - type: main_score - value: 60.65190727391827 - - type: precision - value: 59.61144833427442 - - type: recall - value: 63.537549407114625 - task: - type: BitextMining - - dataset: - config: arb_Latn-rus_Cyrl - name: MTEB FloresBitextMining (arb_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 13.142292490118576 - - type: f1 - value: 12.372910318176764 - - type: main_score - value: 12.372910318176764 - - type: precision - value: 12.197580895919188 - - type: recall - value: 13.142292490118576 - task: - type: BitextMining - - dataset: - config: cat_Latn-rus_Cyrl - name: MTEB FloresBitextMining (cat_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.80599472990777 - - type: main_score - value: 98.80599472990777 - - type: precision - value: 98.72953133822698 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: fur_Latn-rus_Cyrl - name: MTEB FloresBitextMining (fur_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.02766798418972 - - type: f1 - value: 79.36184294084613 - - type: main_score - value: 79.36184294084613 - - type: precision - value: 78.69187826527705 - - type: recall - value: 81.02766798418972 - task: - type: BitextMining - - dataset: - config: kab_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kab_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 34.387351778656125 - - type: f1 - value: 32.02306921576947 - - type: main_score - value: 32.02306921576947 - - type: precision - value: 31.246670347137467 - - type: recall - value: 34.387351778656125 - task: - type: BitextMining - - dataset: - config: lim_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lim_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 78.26086956521739 - - type: f1 - value: 75.90239449214359 - - type: main_score - value: 75.90239449214359 - - type: precision - value: 75.02211430745493 - - type: recall - value: 78.26086956521739 - task: - type: BitextMining - - dataset: - config: nld_Latn-rus_Cyrl - name: MTEB FloresBitextMining (nld_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.9459815546772 - - type: main_score - value: 98.9459815546772 - - type: precision - value: 98.81422924901186 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: san_Deva-rus_Cyrl - name: MTEB FloresBitextMining (san_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 87.94466403162056 - - type: f1 - value: 86.68928897189767 - - type: main_score - value: 86.68928897189767 - - type: precision - value: 86.23822997079216 - - type: recall - value: 87.94466403162056 - task: - type: BitextMining - - dataset: - config: tat_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (tat_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.03557312252964 - - type: f1 - value: 96.4167365353136 - - type: main_score - value: 96.4167365353136 - - type: precision - value: 96.16847826086958 - - type: recall - value: 97.03557312252964 - task: - type: BitextMining - - dataset: - config: xho_Latn-rus_Cyrl - name: MTEB FloresBitextMining (xho_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 86.95652173913044 - - type: f1 - value: 85.5506497283435 - - type: main_score - value: 85.5506497283435 - - type: precision - value: 84.95270479733395 - - type: recall - value: 86.95652173913044 - task: - type: BitextMining - - dataset: - config: ars_Arab-rus_Cyrl - name: MTEB FloresBitextMining (ars_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 96.6403162055336 - - type: f1 - value: 95.60935441370223 - - type: main_score - value: 95.60935441370223 - - type: precision - value: 95.13339920948617 - - type: recall - value: 96.6403162055336 - task: - type: BitextMining - - dataset: - config: ceb_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ceb_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.7509881422925 - - type: f1 - value: 95.05209198303827 - - type: main_score - value: 95.05209198303827 - - type: precision - value: 94.77662283368805 - - type: recall - value: 95.7509881422925 - task: - type: BitextMining - - dataset: - config: fuv_Latn-rus_Cyrl - name: MTEB FloresBitextMining (fuv_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 45.25691699604743 - - type: f1 - value: 42.285666666742365 - - type: main_score - value: 42.285666666742365 - - type: precision - value: 41.21979853402283 - - type: recall - value: 45.25691699604743 - task: - type: BitextMining - - dataset: - config: kac_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kac_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 34.683794466403164 - - type: f1 - value: 33.3235346229031 - - type: main_score - value: 33.3235346229031 - - type: precision - value: 32.94673924616852 - - type: recall - value: 34.683794466403164 - task: - type: BitextMining - - dataset: - config: lin_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lin_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 86.85770750988142 - - type: f1 - value: 85.1867110799439 - - type: main_score - value: 85.1867110799439 - - type: precision - value: 84.53038212173273 - - type: recall - value: 86.85770750988142 - task: - type: BitextMining - - dataset: - config: nno_Latn-rus_Cyrl - name: MTEB FloresBitextMining (nno_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.4308300395257 - - type: f1 - value: 96.78383210991906 - - type: main_score - value: 96.78383210991906 - - type: precision - value: 96.51185770750989 - - type: recall - value: 97.4308300395257 - task: - type: BitextMining - - dataset: - config: sat_Olck-rus_Cyrl - name: MTEB FloresBitextMining (sat_Olck-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 1.185770750988142 - - type: f1 - value: 1.0279253129117258 - - type: main_score - value: 1.0279253129117258 - - type: precision - value: 1.0129746819135175 - - type: recall - value: 1.185770750988142 - task: - type: BitextMining - - dataset: - config: tel_Telu-rus_Cyrl - name: MTEB FloresBitextMining (tel_Telu-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.12252964426878 - - type: f1 - value: 97.61198945981555 - - type: main_score - value: 97.61198945981555 - - type: precision - value: 97.401185770751 - - type: recall - value: 98.12252964426878 - task: - type: BitextMining - - dataset: - config: ydd_Hebr-rus_Cyrl - name: MTEB FloresBitextMining (ydd_Hebr-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 75.8893280632411 - - type: f1 - value: 74.00244008018511 - - type: main_score - value: 74.00244008018511 - - type: precision - value: 73.25683020960382 - - type: recall - value: 75.8893280632411 - task: - type: BitextMining - - dataset: - config: ary_Arab-rus_Cyrl - name: MTEB FloresBitextMining (ary_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 86.56126482213439 - - type: f1 - value: 83.72796285839765 - - type: main_score - value: 83.72796285839765 - - type: precision - value: 82.65014273166447 - - type: recall - value: 86.56126482213439 - task: - type: BitextMining - - dataset: - config: ces_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ces_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.60474308300395 - - type: f1 - value: 99.4729907773386 - - type: main_score - value: 99.4729907773386 - - type: precision - value: 99.40711462450594 - - type: recall - value: 99.60474308300395 - task: - type: BitextMining - - dataset: - config: gaz_Latn-rus_Cyrl - name: MTEB FloresBitextMining (gaz_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 42.58893280632411 - - type: f1 - value: 40.75832866805978 - - type: main_score - value: 40.75832866805978 - - type: precision - value: 40.14285046917723 - - type: recall - value: 42.58893280632411 - task: - type: BitextMining - - dataset: - config: kam_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kam_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 45.25691699604743 - - type: f1 - value: 42.6975518029456 - - type: main_score - value: 42.6975518029456 - - type: precision - value: 41.87472710984596 - - type: recall - value: 45.25691699604743 - task: - type: BitextMining - - dataset: - config: lit_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lit_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.33201581027669 - - type: f1 - value: 96.62384716732542 - - type: main_score - value: 96.62384716732542 - - type: precision - value: 96.3175230566535 - - type: recall - value: 97.33201581027669 - task: - type: BitextMining - - dataset: - config: nob_Latn-rus_Cyrl - name: MTEB FloresBitextMining (nob_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.30368906455863 - - type: main_score - value: 98.30368906455863 - - type: precision - value: 98.10606060606061 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: scn_Latn-rus_Cyrl - name: MTEB FloresBitextMining (scn_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 70.45454545454545 - - type: f1 - value: 68.62561022640075 - - type: main_score - value: 68.62561022640075 - - type: precision - value: 67.95229103411222 - - type: recall - value: 70.45454545454545 - task: - type: BitextMining - - dataset: - config: tgk_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (tgk_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.4901185770751 - - type: f1 - value: 91.58514492753623 - - type: main_score - value: 91.58514492753623 - - type: precision - value: 91.24759298672342 - - type: recall - value: 92.4901185770751 - task: - type: BitextMining - - dataset: - config: yor_Latn-rus_Cyrl - name: MTEB FloresBitextMining (yor_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 67.98418972332016 - - type: f1 - value: 64.72874247330768 - - type: main_score - value: 64.72874247330768 - - type: precision - value: 63.450823399938685 - - type: recall - value: 67.98418972332016 - task: - type: BitextMining - - dataset: - config: arz_Arab-rus_Cyrl - name: MTEB FloresBitextMining (arz_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 94.56521739130434 - - type: f1 - value: 93.07971014492755 - - type: main_score - value: 93.07971014492755 - - type: precision - value: 92.42753623188406 - - type: recall - value: 94.56521739130434 - task: - type: BitextMining - - dataset: - config: cjk_Latn-rus_Cyrl - name: MTEB FloresBitextMining (cjk_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 38.63636363636363 - - type: f1 - value: 36.25747140862938 - - type: main_score - value: 36.25747140862938 - - type: precision - value: 35.49101355074723 - - type: recall - value: 38.63636363636363 - task: - type: BitextMining - - dataset: - config: gla_Latn-rus_Cyrl - name: MTEB FloresBitextMining (gla_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 69.26877470355731 - - type: f1 - value: 66.11797423328613 - - type: main_score - value: 66.11797423328613 - - type: precision - value: 64.89369649409694 - - type: recall - value: 69.26877470355731 - task: - type: BitextMining - - dataset: - config: kan_Knda-rus_Cyrl - name: MTEB FloresBitextMining (kan_Knda-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.51505740636176 - - type: main_score - value: 97.51505740636176 - - type: precision - value: 97.30731225296442 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: lmo_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lmo_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 73.3201581027668 - - type: f1 - value: 71.06371608677273 - - type: main_score - value: 71.06371608677273 - - type: precision - value: 70.26320288266223 - - type: recall - value: 73.3201581027668 - task: - type: BitextMining - - dataset: - config: npi_Deva-rus_Cyrl - name: MTEB FloresBitextMining (npi_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.82608695652173 - - type: f1 - value: 97.36645107198466 - - type: main_score - value: 97.36645107198466 - - type: precision - value: 97.1772068511199 - - type: recall - value: 97.82608695652173 - task: - type: BitextMining - - dataset: - config: shn_Mymr-rus_Cyrl - name: MTEB FloresBitextMining (shn_Mymr-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 39.426877470355734 - - type: f1 - value: 37.16728785513024 - - type: main_score - value: 37.16728785513024 - - type: precision - value: 36.56918548278505 - - type: recall - value: 39.426877470355734 - task: - type: BitextMining - - dataset: - config: tgl_Latn-rus_Cyrl - name: MTEB FloresBitextMining (tgl_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.92490118577075 - - type: f1 - value: 97.6378693769998 - - type: main_score - value: 97.6378693769998 - - type: precision - value: 97.55371440154047 - - type: recall - value: 97.92490118577075 - task: - type: BitextMining - - dataset: - config: yue_Hant-rus_Cyrl - name: MTEB FloresBitextMining (yue_Hant-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.92490118577075 - - type: f1 - value: 97.3833051006964 - - type: main_score - value: 97.3833051006964 - - type: precision - value: 97.1590909090909 - - type: recall - value: 97.92490118577075 - task: - type: BitextMining - - dataset: - config: asm_Beng-rus_Cyrl - name: MTEB FloresBitextMining (asm_Beng-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.78656126482213 - - type: f1 - value: 91.76917395296842 - - type: main_score - value: 91.76917395296842 - - type: precision - value: 91.38292866553736 - - type: recall - value: 92.78656126482213 - task: - type: BitextMining - - dataset: - config: ckb_Arab-rus_Cyrl - name: MTEB FloresBitextMining (ckb_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 80.8300395256917 - - type: f1 - value: 79.17664345468799 - - type: main_score - value: 79.17664345468799 - - type: precision - value: 78.5622171683459 - - type: recall - value: 80.8300395256917 - task: - type: BitextMining - - dataset: - config: gle_Latn-rus_Cyrl - name: MTEB FloresBitextMining (gle_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 85.86956521739131 - - type: f1 - value: 84.45408265372492 - - type: main_score - value: 84.45408265372492 - - type: precision - value: 83.8774340026703 - - type: recall - value: 85.86956521739131 - task: - type: BitextMining - - dataset: - config: kas_Arab-rus_Cyrl - name: MTEB FloresBitextMining (kas_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 76.28458498023716 - - type: f1 - value: 74.11216313578267 - - type: main_score - value: 74.11216313578267 - - type: precision - value: 73.2491277759584 - - type: recall - value: 76.28458498023716 - task: - type: BitextMining - - dataset: - config: ltg_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ltg_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 71.14624505928853 - - type: f1 - value: 68.69245357723618 - - type: main_score - value: 68.69245357723618 - - type: precision - value: 67.8135329666459 - - type: recall - value: 71.14624505928853 - task: - type: BitextMining - - dataset: - config: nso_Latn-rus_Cyrl - name: MTEB FloresBitextMining (nso_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 87.64822134387352 - - type: f1 - value: 85.98419219986725 - - type: main_score - value: 85.98419219986725 - - type: precision - value: 85.32513873917036 - - type: recall - value: 87.64822134387352 - task: - type: BitextMining - - dataset: - config: sin_Sinh-rus_Cyrl - name: MTEB FloresBitextMining (sin_Sinh-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.62845849802372 - - type: f1 - value: 97.10144927536231 - - type: main_score - value: 97.10144927536231 - - type: precision - value: 96.87986585219788 - - type: recall - value: 97.62845849802372 - task: - type: BitextMining - - dataset: - config: tha_Thai-rus_Cyrl - name: MTEB FloresBitextMining (tha_Thai-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.71541501976284 - - type: f1 - value: 98.28722002635045 - - type: main_score - value: 98.28722002635045 - - type: precision - value: 98.07312252964427 - - type: recall - value: 98.71541501976284 - task: - type: BitextMining - - dataset: - config: zho_Hans-rus_Cyrl - name: MTEB FloresBitextMining (zho_Hans-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.68247694334651 - - type: main_score - value: 98.68247694334651 - - type: precision - value: 98.51778656126481 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: ast_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ast_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.65217391304348 - - type: f1 - value: 94.90649683857505 - - type: main_score - value: 94.90649683857505 - - type: precision - value: 94.61352657004831 - - type: recall - value: 95.65217391304348 - task: - type: BitextMining - - dataset: - config: crh_Latn-rus_Cyrl - name: MTEB FloresBitextMining (crh_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 93.08300395256917 - - type: f1 - value: 92.20988998886428 - - type: main_score - value: 92.20988998886428 - - type: precision - value: 91.85631013694254 - - type: recall - value: 93.08300395256917 - task: - type: BitextMining - - dataset: - config: glg_Latn-rus_Cyrl - name: MTEB FloresBitextMining (glg_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.55335968379447 - - type: f1 - value: 95.18006148440931 - - type: main_score - value: 95.18006148440931 - - type: precision - value: 95.06540560888386 - - type: recall - value: 95.55335968379447 - task: - type: BitextMining - - dataset: - config: kas_Deva-rus_Cyrl - name: MTEB FloresBitextMining (kas_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 55.03952569169961 - - type: f1 - value: 52.19871938895554 - - type: main_score - value: 52.19871938895554 - - type: precision - value: 51.17660971469557 - - type: recall - value: 55.03952569169961 - task: - type: BitextMining - - dataset: - config: ltz_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ltz_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 87.64822134387352 - - type: f1 - value: 86.64179841897234 - - type: main_score - value: 86.64179841897234 - - type: precision - value: 86.30023235431587 - - type: recall - value: 87.64822134387352 - task: - type: BitextMining - - dataset: - config: nus_Latn-rus_Cyrl - name: MTEB FloresBitextMining (nus_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 27.4703557312253 - - type: f1 - value: 25.703014277858088 - - type: main_score - value: 25.703014277858088 - - type: precision - value: 25.194105476917315 - - type: recall - value: 27.4703557312253 - task: - type: BitextMining - - dataset: - config: slk_Latn-rus_Cyrl - name: MTEB FloresBitextMining (slk_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.1106719367589 - - type: main_score - value: 99.1106719367589 - - type: precision - value: 99.02832674571805 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: tir_Ethi-rus_Cyrl - name: MTEB FloresBitextMining (tir_Ethi-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 80.73122529644269 - - type: f1 - value: 78.66903754775608 - - type: main_score - value: 78.66903754775608 - - type: precision - value: 77.86431694163612 - - type: recall - value: 80.73122529644269 - task: - type: BitextMining - - dataset: - config: zho_Hant-rus_Cyrl - name: MTEB FloresBitextMining (zho_Hant-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.22134387351778 - - type: f1 - value: 97.66798418972333 - - type: main_score - value: 97.66798418972333 - - type: precision - value: 97.40612648221344 - - type: recall - value: 98.22134387351778 - task: - type: BitextMining - - dataset: - config: awa_Deva-rus_Cyrl - name: MTEB FloresBitextMining (awa_Deva-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.5296442687747 - - type: f1 - value: 96.94224857268335 - - type: main_score - value: 96.94224857268335 - - type: precision - value: 96.68560606060606 - - type: recall - value: 97.5296442687747 - task: - type: BitextMining - - dataset: - config: cym_Latn-rus_Cyrl - name: MTEB FloresBitextMining (cym_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 92.68774703557312 - - type: f1 - value: 91.69854302097961 - - type: main_score - value: 91.69854302097961 - - type: precision - value: 91.31236846157795 - - type: recall - value: 92.68774703557312 - task: - type: BitextMining - - dataset: - config: grn_Latn-rus_Cyrl - name: MTEB FloresBitextMining (grn_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 64.13043478260869 - - type: f1 - value: 61.850586118740004 - - type: main_score - value: 61.850586118740004 - - type: precision - value: 61.0049495186209 - - type: recall - value: 64.13043478260869 - task: - type: BitextMining - - dataset: - config: kat_Geor-rus_Cyrl - name: MTEB FloresBitextMining (kat_Geor-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.59881422924902 - - type: main_score - value: 97.59881422924902 - - type: precision - value: 97.42534036012296 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: lua_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lua_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 63.63636363636363 - - type: f1 - value: 60.9709122526128 - - type: main_score - value: 60.9709122526128 - - type: precision - value: 60.03915902282226 - - type: recall - value: 63.63636363636363 - task: - type: BitextMining - - dataset: - config: nya_Latn-rus_Cyrl - name: MTEB FloresBitextMining (nya_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 89.2292490118577 - - type: f1 - value: 87.59723824473149 - - type: main_score - value: 87.59723824473149 - - type: precision - value: 86.90172707867349 - - type: recall - value: 89.2292490118577 - task: - type: BitextMining - - dataset: - config: slv_Latn-rus_Cyrl - name: MTEB FloresBitextMining (slv_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.01185770750988 - - type: f1 - value: 98.74835309617917 - - type: main_score - value: 98.74835309617917 - - type: precision - value: 98.63636363636364 - - type: recall - value: 99.01185770750988 - task: - type: BitextMining - - dataset: - config: tpi_Latn-rus_Cyrl - name: MTEB FloresBitextMining (tpi_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 77.37154150197628 - - type: f1 - value: 75.44251611276084 - - type: main_score - value: 75.44251611276084 - - type: precision - value: 74.78103665109595 - - type: recall - value: 77.37154150197628 - task: - type: BitextMining - - dataset: - config: zsm_Latn-rus_Cyrl - name: MTEB FloresBitextMining (zsm_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.2094861660079 - - type: f1 - value: 98.96245059288538 - - type: main_score - value: 98.96245059288538 - - type: precision - value: 98.8471673254282 - - type: recall - value: 99.2094861660079 - task: - type: BitextMining - - dataset: - config: ayr_Latn-rus_Cyrl - name: MTEB FloresBitextMining (ayr_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 27.766798418972332 - - type: f1 - value: 26.439103195281312 - - type: main_score - value: 26.439103195281312 - - type: precision - value: 26.052655604573964 - - type: recall - value: 27.766798418972332 - task: - type: BitextMining - - dataset: - config: dan_Latn-rus_Cyrl - name: MTEB FloresBitextMining (dan_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.30830039525692 - - type: f1 - value: 99.07773386034255 - - type: main_score - value: 99.07773386034255 - - type: precision - value: 98.96245059288538 - - type: recall - value: 99.30830039525692 - task: - type: BitextMining - - dataset: - config: guj_Gujr-rus_Cyrl - name: MTEB FloresBitextMining (guj_Gujr-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.82608695652173 - - type: f1 - value: 97.26449275362317 - - type: main_score - value: 97.26449275362317 - - type: precision - value: 97.02498588368154 - - type: recall - value: 97.82608695652173 - task: - type: BitextMining - - dataset: - config: kaz_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (kaz_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.5296442687747 - - type: f1 - value: 97.03557312252964 - - type: main_score - value: 97.03557312252964 - - type: precision - value: 96.85022158342316 - - type: recall - value: 97.5296442687747 - task: - type: BitextMining - - dataset: - config: lug_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lug_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 68.57707509881423 - - type: f1 - value: 65.93361605820395 - - type: main_score - value: 65.93361605820395 - - type: precision - value: 64.90348248593789 - - type: recall - value: 68.57707509881423 - task: - type: BitextMining - - dataset: - config: oci_Latn-rus_Cyrl - name: MTEB FloresBitextMining (oci_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 86.26482213438736 - - type: f1 - value: 85.33176417155623 - - type: main_score - value: 85.33176417155623 - - type: precision - value: 85.00208833384637 - - type: recall - value: 86.26482213438736 - task: - type: BitextMining - - dataset: - config: smo_Latn-rus_Cyrl - name: MTEB FloresBitextMining (smo_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 77.96442687747036 - - type: f1 - value: 75.70960450188885 - - type: main_score - value: 75.70960450188885 - - type: precision - value: 74.8312632736777 - - type: recall - value: 77.96442687747036 - task: - type: BitextMining - - dataset: - config: tsn_Latn-rus_Cyrl - name: MTEB FloresBitextMining (tsn_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 84.38735177865613 - - type: f1 - value: 82.13656376349225 - - type: main_score - value: 82.13656376349225 - - type: precision - value: 81.16794543904518 - - type: recall - value: 84.38735177865613 - task: - type: BitextMining - - dataset: - config: zul_Latn-rus_Cyrl - name: MTEB FloresBitextMining (zul_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 90.21739130434783 - - type: f1 - value: 88.77570602050753 - - type: main_score - value: 88.77570602050753 - - type: precision - value: 88.15978104021582 - - type: recall - value: 90.21739130434783 - task: - type: BitextMining - - dataset: - config: azb_Arab-rus_Cyrl - name: MTEB FloresBitextMining (azb_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 65.71146245059289 - - type: f1 - value: 64.18825390221271 - - type: main_score - value: 64.18825390221271 - - type: precision - value: 63.66811154793568 - - type: recall - value: 65.71146245059289 - task: - type: BitextMining - - dataset: - config: deu_Latn-rus_Cyrl - name: MTEB FloresBitextMining (deu_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 99.70355731225297 - - type: f1 - value: 99.60474308300395 - - type: main_score - value: 99.60474308300395 - - type: precision - value: 99.55533596837944 - - type: recall - value: 99.70355731225297 - task: - type: BitextMining - - dataset: - config: hat_Latn-rus_Cyrl - name: MTEB FloresBitextMining (hat_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 86.7588932806324 - - type: f1 - value: 85.86738623695146 - - type: main_score - value: 85.86738623695146 - - type: precision - value: 85.55235467420822 - - type: recall - value: 86.7588932806324 - task: - type: BitextMining - - dataset: - config: kbp_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kbp_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 34.88142292490119 - - type: f1 - value: 32.16511669463015 - - type: main_score - value: 32.16511669463015 - - type: precision - value: 31.432098549546318 - - type: recall - value: 34.88142292490119 - task: - type: BitextMining - - dataset: - config: luo_Latn-rus_Cyrl - name: MTEB FloresBitextMining (luo_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 52.27272727272727 - - type: f1 - value: 49.60489626836975 - - type: main_score - value: 49.60489626836975 - - type: precision - value: 48.69639631803339 - - type: recall - value: 52.27272727272727 - task: - type: BitextMining - - dataset: - config: ory_Orya-rus_Cyrl - name: MTEB FloresBitextMining (ory_Orya-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.82608695652173 - - type: f1 - value: 97.27437417654808 - - type: main_score - value: 97.27437417654808 - - type: precision - value: 97.04968944099377 - - type: recall - value: 97.82608695652173 - task: - type: BitextMining - - dataset: - config: sna_Latn-rus_Cyrl - name: MTEB FloresBitextMining (sna_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 85.37549407114624 - - type: f1 - value: 83.09911316305177 - - type: main_score - value: 83.09911316305177 - - type: precision - value: 82.1284950958864 - - type: recall - value: 85.37549407114624 - task: - type: BitextMining - - dataset: - config: tso_Latn-rus_Cyrl - name: MTEB FloresBitextMining (tso_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 82.90513833992095 - - type: f1 - value: 80.28290385503824 - - type: main_score - value: 80.28290385503824 - - type: precision - value: 79.23672543237761 - - type: recall - value: 82.90513833992095 - task: - type: BitextMining - - dataset: - config: azj_Latn-rus_Cyrl - name: MTEB FloresBitextMining (azj_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.02371541501977 - - type: f1 - value: 97.49200075287031 - - type: main_score - value: 97.49200075287031 - - type: precision - value: 97.266139657444 - - type: recall - value: 98.02371541501977 - task: - type: BitextMining - - dataset: - config: dik_Latn-rus_Cyrl - name: MTEB FloresBitextMining (dik_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 38.43873517786561 - - type: f1 - value: 35.78152442955223 - - type: main_score - value: 35.78152442955223 - - type: precision - value: 34.82424325078237 - - type: recall - value: 38.43873517786561 - task: - type: BitextMining - - dataset: - config: hau_Latn-rus_Cyrl - name: MTEB FloresBitextMining (hau_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.42292490118577 - - type: f1 - value: 79.24612283124593 - - type: main_score - value: 79.24612283124593 - - type: precision - value: 78.34736070751448 - - type: recall - value: 81.42292490118577 - task: - type: BitextMining - - dataset: - config: kea_Latn-rus_Cyrl - name: MTEB FloresBitextMining (kea_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 81.62055335968378 - - type: f1 - value: 80.47015182884748 - - type: main_score - value: 80.47015182884748 - - type: precision - value: 80.02671028885862 - - type: recall - value: 81.62055335968378 - task: - type: BitextMining - - dataset: - config: lus_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lus_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 62.74703557312253 - - type: f1 - value: 60.53900079111122 - - type: main_score - value: 60.53900079111122 - - type: precision - value: 59.80024202850289 - - type: recall - value: 62.74703557312253 - task: - type: BitextMining - - dataset: - config: pag_Latn-rus_Cyrl - name: MTEB FloresBitextMining (pag_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 74.01185770750988 - - type: f1 - value: 72.57280648279529 - - type: main_score - value: 72.57280648279529 - - type: precision - value: 71.99952968456789 - - type: recall - value: 74.01185770750988 - task: - type: BitextMining - - dataset: - config: snd_Arab-rus_Cyrl - name: MTEB FloresBitextMining (snd_Arab-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 91.30434782608695 - - type: f1 - value: 90.24653499445358 - - type: main_score - value: 90.24653499445358 - - type: precision - value: 89.83134068200232 - - type: recall - value: 91.30434782608695 - task: - type: BitextMining - - dataset: - config: tuk_Latn-rus_Cyrl - name: MTEB FloresBitextMining (tuk_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 47.62845849802372 - - type: f1 - value: 45.812928836644254 - - type: main_score - value: 45.812928836644254 - - type: precision - value: 45.23713833170355 - - type: recall - value: 47.62845849802372 - task: - type: BitextMining - - dataset: - config: bak_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (bak_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.8498023715415 - - type: f1 - value: 95.18904459615922 - - type: main_score - value: 95.18904459615922 - - type: precision - value: 94.92812441182006 - - type: recall - value: 95.8498023715415 - task: - type: BitextMining - - dataset: - config: dyu_Latn-rus_Cyrl - name: MTEB FloresBitextMining (dyu_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 29.64426877470356 - - type: f1 - value: 27.287335193938166 - - type: main_score - value: 27.287335193938166 - - type: precision - value: 26.583996026587492 - - type: recall - value: 29.64426877470356 - task: - type: BitextMining - - dataset: - config: heb_Hebr-rus_Cyrl - name: MTEB FloresBitextMining (heb_Hebr-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 98.91304347826086 - - type: f1 - value: 98.55072463768116 - - type: main_score - value: 98.55072463768116 - - type: precision - value: 98.36956521739131 - - type: recall - value: 98.91304347826086 - task: - type: BitextMining - - dataset: - config: khk_Cyrl-rus_Cyrl - name: MTEB FloresBitextMining (khk_Cyrl-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 95.15810276679841 - - type: f1 - value: 94.44009547764487 - - type: main_score - value: 94.44009547764487 - - type: precision - value: 94.16579797014579 - - type: recall - value: 95.15810276679841 - task: - type: BitextMining - - dataset: - config: lvs_Latn-rus_Cyrl - name: MTEB FloresBitextMining (lvs_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.92490118577075 - - type: f1 - value: 97.51467241585817 - - type: main_score - value: 97.51467241585817 - - type: precision - value: 97.36166007905138 - - type: recall - value: 97.92490118577075 - task: - type: BitextMining - - dataset: - config: pan_Guru-rus_Cyrl - name: MTEB FloresBitextMining (pan_Guru-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 97.92490118577075 - - type: f1 - value: 97.42918313570486 - - type: main_score - value: 97.42918313570486 - - type: precision - value: 97.22261434217955 - - type: recall - value: 97.92490118577075 - task: - type: BitextMining - - dataset: - config: som_Latn-rus_Cyrl - name: MTEB FloresBitextMining (som_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 75.69169960474308 - - type: f1 - value: 73.7211667065916 - - type: main_score - value: 73.7211667065916 - - type: precision - value: 72.95842401892384 - - type: recall - value: 75.69169960474308 - task: - type: BitextMining - - dataset: - config: tum_Latn-rus_Cyrl - name: MTEB FloresBitextMining (tum_Latn-rus_Cyrl) - revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e - split: devtest - type: mteb/flores - metrics: - - type: accuracy - value: 85.67193675889328 - - type: f1 - value: 82.9296066252588 - - type: main_score - value: 82.9296066252588 - - type: precision - value: 81.77330225447936 - - type: recall - value: 85.67193675889328 - task: - type: BitextMining - - dataset: - config: default - name: MTEB GeoreviewClassification (default) - revision: 3765c0d1de6b7d264bc459433c45e5a75513839c - split: test - type: ai-forever/georeview-classification - metrics: - - type: accuracy - value: 44.6630859375 - - type: f1 - value: 42.607425073610536 - - type: f1_weighted - value: 42.60639474586065 - - type: main_score - value: 44.6630859375 - task: - type: Classification - - dataset: - config: default - name: MTEB GeoreviewClusteringP2P (default) - revision: 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec - split: test - type: ai-forever/georeview-clustering-p2p - metrics: - - type: main_score - value: 58.15951247070825 - - type: v_measure - value: 58.15951247070825 - - type: v_measure_std - value: 0.6739615788288809 - task: - type: Clustering - - dataset: - config: default - name: MTEB HeadlineClassification (default) - revision: 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb - split: test - type: ai-forever/headline-classification - metrics: - - type: accuracy - value: 73.935546875 - - type: f1 - value: 73.8654872186846 - - type: f1_weighted - value: 73.86733122685095 - - type: main_score - value: 73.935546875 - task: - type: Classification - - dataset: - config: default - name: MTEB InappropriatenessClassification (default) - revision: 601651fdc45ef243751676e62dd7a19f491c0285 - split: test - type: ai-forever/inappropriateness-classification - metrics: - - type: accuracy - value: 59.16015624999999 - - type: ap - value: 55.52276605836938 - - type: ap_weighted - value: 55.52276605836938 - - type: f1 - value: 58.614248199637956 - - type: f1_weighted - value: 58.614248199637956 - - type: main_score - value: 59.16015624999999 - task: - type: Classification - - dataset: - config: default - name: MTEB KinopoiskClassification (default) - revision: 5911f26666ac11af46cb9c6849d0dc80a378af24 - split: test - type: ai-forever/kinopoisk-sentiment-classification - metrics: - - type: accuracy - value: 49.959999999999994 - - type: f1 - value: 48.4900332316098 - - type: f1_weighted - value: 48.4900332316098 - - type: main_score - value: 49.959999999999994 - task: - type: Classification - - dataset: - config: default - name: MTEB LanguageClassification (default) - revision: aa56583bf2bc52b0565770607d6fc3faebecf9e2 - split: test - type: papluca/language-identification - metrics: - - type: accuracy - value: 71.005859375 - - type: f1 - value: 69.63481100303348 - - type: f1_weighted - value: 69.64640413409529 - - type: main_score - value: 71.005859375 - task: - type: Classification - - dataset: - config: ru - name: MTEB MLSUMClusteringP2P (ru) - revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7 - split: test - type: reciTAL/mlsum - metrics: - - type: main_score - value: 42.11280087032343 - - type: v_measure - value: 42.11280087032343 - - type: v_measure_std - value: 6.7619971723605135 - task: - type: Clustering - - dataset: - config: ru - name: MTEB MLSUMClusteringP2P.v2 (ru) - revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7 - split: test - type: reciTAL/mlsum - metrics: - - type: main_score - value: 43.00112546945811 - - type: v_measure - value: 43.00112546945811 - - type: v_measure_std - value: 1.4740560414835675 - task: - type: Clustering - - dataset: - config: ru - name: MTEB MLSUMClusteringS2S (ru) - revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7 - split: test - type: reciTAL/mlsum - metrics: - - type: main_score - value: 39.81446080575161 - - type: v_measure - value: 39.81446080575161 - - type: v_measure_std - value: 7.125661320308298 - task: - type: Clustering - - dataset: - config: ru - name: MTEB MLSUMClusteringS2S.v2 (ru) - revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7 - split: test - type: reciTAL/mlsum - metrics: - - type: main_score - value: 39.29659668980239 - - type: v_measure - value: 39.29659668980239 - - type: v_measure_std - value: 2.6570502923023094 - task: - type: Clustering - - dataset: - config: ru - name: MTEB MultiLongDocRetrieval (ru) - revision: d67138e705d963e346253a80e59676ddb418810a - split: dev - type: Shitao/MLDR - metrics: - - type: main_score - value: 38.671 - - type: map_at_1 - value: 30.0 - - type: map_at_10 - value: 36.123 - - type: map_at_100 - value: 36.754999999999995 - - type: map_at_1000 - value: 36.806 - - type: map_at_20 - value: 36.464 - - type: map_at_3 - value: 35.25 - - type: map_at_5 - value: 35.8 - - type: mrr_at_1 - value: 30.0 - - type: mrr_at_10 - value: 36.122817460317464 - - type: mrr_at_100 - value: 36.75467016625293 - - type: mrr_at_1000 - value: 36.80612724920882 - - type: mrr_at_20 - value: 36.46359681984682 - - type: mrr_at_3 - value: 35.25 - - type: mrr_at_5 - value: 35.800000000000004 - - type: nauc_map_at_1000_diff1 - value: 55.61987610843598 - - type: nauc_map_at_1000_max - value: 52.506795017152186 - - type: nauc_map_at_1000_std - value: 2.95487192066911 - - type: nauc_map_at_100_diff1 - value: 55.598419532054734 - - type: nauc_map_at_100_max - value: 52.48192017040307 - - type: nauc_map_at_100_std - value: 2.930120252521189 - - type: nauc_map_at_10_diff1 - value: 56.02309155375198 - - type: nauc_map_at_10_max - value: 52.739573233234424 - - type: nauc_map_at_10_std - value: 2.4073432421641545 - - type: nauc_map_at_1_diff1 - value: 52.57059856776112 - - type: nauc_map_at_1_max - value: 50.55668152952304 - - type: nauc_map_at_1_std - value: 1.6572084853398048 - - type: nauc_map_at_20_diff1 - value: 55.75769029917031 - - type: nauc_map_at_20_max - value: 52.53663737242853 - - type: nauc_map_at_20_std - value: 2.8489192879814 - - type: nauc_map_at_3_diff1 - value: 56.90294128342709 - - type: nauc_map_at_3_max - value: 53.10608389782041 - - type: nauc_map_at_3_std - value: 1.4909731657889491 - - type: nauc_map_at_5_diff1 - value: 56.1258315436073 - - type: nauc_map_at_5_max - value: 52.398078357541564 - - type: nauc_map_at_5_std - value: 1.8256862015101467 - - type: nauc_mrr_at_1000_diff1 - value: 55.61987610843598 - - type: nauc_mrr_at_1000_max - value: 52.506795017152186 - - type: nauc_mrr_at_1000_std - value: 2.95487192066911 - - type: nauc_mrr_at_100_diff1 - value: 55.598419532054734 - - type: nauc_mrr_at_100_max - value: 52.48192017040307 - - type: nauc_mrr_at_100_std - value: 2.930120252521189 - - type: nauc_mrr_at_10_diff1 - value: 56.02309155375198 - - type: nauc_mrr_at_10_max - value: 52.739573233234424 - - type: nauc_mrr_at_10_std - value: 2.4073432421641545 - - type: nauc_mrr_at_1_diff1 - value: 52.57059856776112 - - type: nauc_mrr_at_1_max - value: 50.55668152952304 - - type: nauc_mrr_at_1_std - value: 1.6572084853398048 - - type: nauc_mrr_at_20_diff1 - value: 55.75769029917031 - - type: nauc_mrr_at_20_max - value: 52.53663737242853 - - type: nauc_mrr_at_20_std - value: 2.8489192879814 - - type: nauc_mrr_at_3_diff1 - value: 56.90294128342709 - - type: nauc_mrr_at_3_max - value: 53.10608389782041 - - type: nauc_mrr_at_3_std - value: 1.4909731657889491 - - type: nauc_mrr_at_5_diff1 - value: 56.1258315436073 - - type: nauc_mrr_at_5_max - value: 52.398078357541564 - - type: nauc_mrr_at_5_std - value: 1.8256862015101467 - - type: nauc_ndcg_at_1000_diff1 - value: 55.30733548408918 - - type: nauc_ndcg_at_1000_max - value: 53.51143366189318 - - type: nauc_ndcg_at_1000_std - value: 7.133789405525702 - - type: nauc_ndcg_at_100_diff1 - value: 54.32209039488095 - - type: nauc_ndcg_at_100_max - value: 52.67499334461009 - - type: nauc_ndcg_at_100_std - value: 6.878823275077807 - - type: nauc_ndcg_at_10_diff1 - value: 56.266780806997716 - - type: nauc_ndcg_at_10_max - value: 53.52837255793743 - - type: nauc_ndcg_at_10_std - value: 3.756832592964262 - - type: nauc_ndcg_at_1_diff1 - value: 52.57059856776112 - - type: nauc_ndcg_at_1_max - value: 50.55668152952304 - - type: nauc_ndcg_at_1_std - value: 1.6572084853398048 - - type: nauc_ndcg_at_20_diff1 - value: 55.39255420432796 - - type: nauc_ndcg_at_20_max - value: 52.946114684072235 - - type: nauc_ndcg_at_20_std - value: 5.414933414031693 - - type: nauc_ndcg_at_3_diff1 - value: 57.92826624996289 - - type: nauc_ndcg_at_3_max - value: 53.89907760306972 - - type: nauc_ndcg_at_3_std - value: 1.6661401245309218 - - type: nauc_ndcg_at_5_diff1 - value: 56.47508936029308 - - type: nauc_ndcg_at_5_max - value: 52.66800998045517 - - type: nauc_ndcg_at_5_std - value: 2.4127296184140423 - - type: nauc_precision_at_1000_diff1 - value: 57.25924020238401 - - type: nauc_precision_at_1000_max - value: 65.1132590931922 - - type: nauc_precision_at_1000_std - value: 40.60788709618145 - - type: nauc_precision_at_100_diff1 - value: 46.49620002554606 - - type: nauc_precision_at_100_max - value: 53.02960148167071 - - type: nauc_precision_at_100_std - value: 28.206028867032863 - - type: nauc_precision_at_10_diff1 - value: 56.562744749606765 - - type: nauc_precision_at_10_max - value: 56.00594967783547 - - type: nauc_precision_at_10_std - value: 8.368379831645163 - - type: nauc_precision_at_1_diff1 - value: 52.57059856776112 - - type: nauc_precision_at_1_max - value: 50.55668152952304 - - type: nauc_precision_at_1_std - value: 1.6572084853398048 - - type: nauc_precision_at_20_diff1 - value: 53.25915754614111 - - type: nauc_precision_at_20_max - value: 54.03255118937036 - - type: nauc_precision_at_20_std - value: 15.161611674272718 - - type: nauc_precision_at_3_diff1 - value: 60.726785748943854 - - type: nauc_precision_at_3_max - value: 56.139896875869354 - - type: nauc_precision_at_3_std - value: 2.2306901035769893 - - type: nauc_precision_at_5_diff1 - value: 57.1201127525187 - - type: nauc_precision_at_5_max - value: 53.28665761862506 - - type: nauc_precision_at_5_std - value: 4.358720050112237 - - type: nauc_recall_at_1000_diff1 - value: 57.259240202383964 - - type: nauc_recall_at_1000_max - value: 65.11325909319218 - - type: nauc_recall_at_1000_std - value: 40.60788709618142 - - type: nauc_recall_at_100_diff1 - value: 46.49620002554603 - - type: nauc_recall_at_100_max - value: 53.02960148167071 - - type: nauc_recall_at_100_std - value: 28.206028867032835 - - type: nauc_recall_at_10_diff1 - value: 56.562744749606765 - - type: nauc_recall_at_10_max - value: 56.00594967783549 - - type: nauc_recall_at_10_std - value: 8.368379831645147 - - type: nauc_recall_at_1_diff1 - value: 52.57059856776112 - - type: nauc_recall_at_1_max - value: 50.55668152952304 - - type: nauc_recall_at_1_std - value: 1.6572084853398048 - - type: nauc_recall_at_20_diff1 - value: 53.259157546141154 - - type: nauc_recall_at_20_max - value: 54.03255118937038 - - type: nauc_recall_at_20_std - value: 15.16161167427274 - - type: nauc_recall_at_3_diff1 - value: 60.72678574894387 - - type: nauc_recall_at_3_max - value: 56.13989687586933 - - type: nauc_recall_at_3_std - value: 2.2306901035770066 - - type: nauc_recall_at_5_diff1 - value: 57.12011275251864 - - type: nauc_recall_at_5_max - value: 53.28665761862502 - - type: nauc_recall_at_5_std - value: 4.3587200501122245 - - type: ndcg_at_1 - value: 30.0 - - type: ndcg_at_10 - value: 38.671 - - type: ndcg_at_100 - value: 42.173 - - type: ndcg_at_1000 - value: 44.016 - - type: ndcg_at_20 - value: 39.845000000000006 - - type: ndcg_at_3 - value: 36.863 - - type: ndcg_at_5 - value: 37.874 - - type: precision_at_1 - value: 30.0 - - type: precision_at_10 - value: 4.65 - - type: precision_at_100 - value: 0.64 - - type: precision_at_1000 - value: 0.08 - - type: precision_at_20 - value: 2.55 - - type: precision_at_3 - value: 13.833 - - type: precision_at_5 - value: 8.799999999999999 - - type: recall_at_1 - value: 30.0 - - type: recall_at_10 - value: 46.5 - - type: recall_at_100 - value: 64.0 - - type: recall_at_1000 - value: 79.5 - - type: recall_at_20 - value: 51.0 - - type: recall_at_3 - value: 41.5 - - type: recall_at_5 - value: 44.0 - task: - type: Retrieval - - dataset: - config: rus - name: MTEB MultilingualSentimentClassification (rus) - revision: 2b9b4d10fc589af67794141fe8cbd3739de1eb33 - split: test - type: mteb/multilingual-sentiment-classification - metrics: - - type: accuracy - value: 79.52710495963092 - - type: ap - value: 84.5713457178972 - - type: ap_weighted - value: 84.5713457178972 - - type: f1 - value: 77.88661181524105 - - type: f1_weighted - value: 79.87563079922718 - - type: main_score - value: 79.52710495963092 - task: - type: Classification - - dataset: - config: arb_Arab-rus_Cyrl - name: MTEB NTREXBitextMining (arb_Arab-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 86.47971957936905 - - type: f1 - value: 82.79864240805654 - - type: main_score - value: 82.79864240805654 - - type: precision - value: 81.21485800128767 - - type: recall - value: 86.47971957936905 - task: - type: BitextMining - - dataset: - config: bel_Cyrl-rus_Cyrl - name: MTEB NTREXBitextMining (bel_Cyrl-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.84226339509264 - - type: f1 - value: 93.56399067465667 - - type: main_score - value: 93.56399067465667 - - type: precision - value: 93.01619095309631 - - type: recall - value: 94.84226339509264 - task: - type: BitextMining - - dataset: - config: ben_Beng-rus_Cyrl - name: MTEB NTREXBitextMining (ben_Beng-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 92.18828242363544 - - type: f1 - value: 90.42393889620612 - - type: main_score - value: 90.42393889620612 - - type: precision - value: 89.67904925153297 - - type: recall - value: 92.18828242363544 - task: - type: BitextMining - - dataset: - config: bos_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (bos_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.69203805708563 - - type: f1 - value: 93.37172425304624 - - type: main_score - value: 93.37172425304624 - - type: precision - value: 92.79204521067315 - - type: recall - value: 94.69203805708563 - task: - type: BitextMining - - dataset: - config: bul_Cyrl-rus_Cyrl - name: MTEB NTREXBitextMining (bul_Cyrl-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.99549323985978 - - type: f1 - value: 96.13086296110833 - - type: main_score - value: 96.13086296110833 - - type: precision - value: 95.72441996327827 - - type: recall - value: 96.99549323985978 - task: - type: BitextMining - - dataset: - config: ces_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (ces_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.94391587381071 - - type: f1 - value: 94.90680465142157 - - type: main_score - value: 94.90680465142157 - - type: precision - value: 94.44541812719079 - - type: recall - value: 95.94391587381071 - task: - type: BitextMining - - dataset: - config: deu_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (deu_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.09414121181773 - - type: f1 - value: 94.94408279085295 - - type: main_score - value: 94.94408279085295 - - type: precision - value: 94.41245201135037 - - type: recall - value: 96.09414121181773 - task: - type: BitextMining - - dataset: - config: ell_Grek-rus_Cyrl - name: MTEB NTREXBitextMining (ell_Grek-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.19429143715573 - - type: f1 - value: 95.12101485561676 - - type: main_score - value: 95.12101485561676 - - type: precision - value: 94.60440660991488 - - type: recall - value: 96.19429143715573 - task: - type: BitextMining - - dataset: - config: eng_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (eng_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.49474211316975 - - type: f1 - value: 95.46581777428045 - - type: main_score - value: 95.46581777428045 - - type: precision - value: 94.98414288098814 - - type: recall - value: 96.49474211316975 - task: - type: BitextMining - - dataset: - config: fas_Arab-rus_Cyrl - name: MTEB NTREXBitextMining (fas_Arab-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.44166249374061 - - type: f1 - value: 92.92383018972905 - - type: main_score - value: 92.92383018972905 - - type: precision - value: 92.21957936905358 - - type: recall - value: 94.44166249374061 - task: - type: BitextMining - - dataset: - config: fin_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (fin_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 92.18828242363544 - - type: f1 - value: 90.2980661468393 - - type: main_score - value: 90.2980661468393 - - type: precision - value: 89.42580537472877 - - type: recall - value: 92.18828242363544 - task: - type: BitextMining - - dataset: - config: fra_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (fra_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.84376564847271 - - type: f1 - value: 94.81054915706895 - - type: main_score - value: 94.81054915706895 - - type: precision - value: 94.31369276136427 - - type: recall - value: 95.84376564847271 - task: - type: BitextMining - - dataset: - config: heb_Hebr-rus_Cyrl - name: MTEB NTREXBitextMining (heb_Hebr-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.89233850776164 - - type: f1 - value: 93.42513770655985 - - type: main_score - value: 93.42513770655985 - - type: precision - value: 92.73493573693875 - - type: recall - value: 94.89233850776164 - task: - type: BitextMining - - dataset: - config: hin_Deva-rus_Cyrl - name: MTEB NTREXBitextMining (hin_Deva-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.23985978968453 - - type: f1 - value: 91.52816526376867 - - type: main_score - value: 91.52816526376867 - - type: precision - value: 90.76745946425466 - - type: recall - value: 93.23985978968453 - task: - type: BitextMining - - dataset: - config: hrv_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (hrv_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.99098647971958 - - type: f1 - value: 92.36354531797697 - - type: main_score - value: 92.36354531797697 - - type: precision - value: 91.63228970439788 - - type: recall - value: 93.99098647971958 - task: - type: BitextMining - - dataset: - config: hun_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (hun_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.64046069103655 - - type: f1 - value: 92.05224503421799 - - type: main_score - value: 92.05224503421799 - - type: precision - value: 91.33998616973079 - - type: recall - value: 93.64046069103655 - task: - type: BitextMining - - dataset: - config: ind_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (ind_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 91.68753129694541 - - type: f1 - value: 89.26222667334335 - - type: main_score - value: 89.26222667334335 - - type: precision - value: 88.14638624603572 - - type: recall - value: 91.68753129694541 - task: - type: BitextMining - - dataset: - config: jpn_Jpan-rus_Cyrl - name: MTEB NTREXBitextMining (jpn_Jpan-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 91.28693039559339 - - type: f1 - value: 89.21161763348957 - - type: main_score - value: 89.21161763348957 - - type: precision - value: 88.31188340952988 - - type: recall - value: 91.28693039559339 - task: - type: BitextMining - - dataset: - config: kor_Hang-rus_Cyrl - name: MTEB NTREXBitextMining (kor_Hang-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 89.53430145217827 - - type: f1 - value: 86.88322165788365 - - type: main_score - value: 86.88322165788365 - - type: precision - value: 85.73950211030831 - - type: recall - value: 89.53430145217827 - task: - type: BitextMining - - dataset: - config: lit_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (lit_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 90.28542814221332 - - type: f1 - value: 88.10249103814452 - - type: main_score - value: 88.10249103814452 - - type: precision - value: 87.17689323973752 - - type: recall - value: 90.28542814221332 - task: - type: BitextMining - - dataset: - config: mkd_Cyrl-rus_Cyrl - name: MTEB NTREXBitextMining (mkd_Cyrl-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.04256384576865 - - type: f1 - value: 93.65643703650713 - - type: main_score - value: 93.65643703650713 - - type: precision - value: 93.02036387915207 - - type: recall - value: 95.04256384576865 - task: - type: BitextMining - - dataset: - config: nld_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (nld_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.39308963445168 - - type: f1 - value: 94.16207644800535 - - type: main_score - value: 94.16207644800535 - - type: precision - value: 93.582516632091 - - type: recall - value: 95.39308963445168 - task: - type: BitextMining - - dataset: - config: pol_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (pol_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.7436154231347 - - type: f1 - value: 94.5067601402103 - - type: main_score - value: 94.5067601402103 - - type: precision - value: 93.91587381071608 - - type: recall - value: 95.7436154231347 - task: - type: BitextMining - - dataset: - config: por_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (por_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 65.89884827240861 - - type: f1 - value: 64.61805459419219 - - type: main_score - value: 64.61805459419219 - - type: precision - value: 64.07119451106485 - - type: recall - value: 65.89884827240861 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-arb_Arab - name: MTEB NTREXBitextMining (rus_Cyrl-arb_Arab) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.2413620430646 - - type: f1 - value: 92.67663399861698 - - type: main_score - value: 92.67663399861698 - - type: precision - value: 91.94625271240193 - - type: recall - value: 94.2413620430646 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bel_Cyrl - name: MTEB NTREXBitextMining (rus_Cyrl-bel_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.89233850776164 - - type: f1 - value: 93.40343849106993 - - type: main_score - value: 93.40343849106993 - - type: precision - value: 92.74077783341679 - - type: recall - value: 94.89233850776164 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ben_Beng - name: MTEB NTREXBitextMining (rus_Cyrl-ben_Beng) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.2914371557336 - - type: f1 - value: 92.62226673343348 - - type: main_score - value: 92.62226673343348 - - type: precision - value: 91.84610248706393 - - type: recall - value: 94.2914371557336 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bos_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-bos_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.69354031046569 - - type: f1 - value: 94.50418051319403 - - type: main_score - value: 94.50418051319403 - - type: precision - value: 93.95843765648473 - - type: recall - value: 95.69354031046569 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-bul_Cyrl - name: MTEB NTREXBitextMining (rus_Cyrl-bul_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.89384076114172 - - type: f1 - value: 94.66199298948423 - - type: main_score - value: 94.66199298948423 - - type: precision - value: 94.08028709731263 - - type: recall - value: 95.89384076114172 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ces_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-ces_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.94091136705057 - - type: f1 - value: 92.3746731207923 - - type: main_score - value: 92.3746731207923 - - type: precision - value: 91.66207644800535 - - type: recall - value: 93.94091136705057 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-deu_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-deu_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.94391587381071 - - type: f1 - value: 94.76214321482223 - - type: main_score - value: 94.76214321482223 - - type: precision - value: 94.20380570856285 - - type: recall - value: 95.94391587381071 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ell_Grek - name: MTEB NTREXBitextMining (rus_Cyrl-ell_Grek) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.44316474712068 - - type: f1 - value: 94.14788849941579 - - type: main_score - value: 94.14788849941579 - - type: precision - value: 93.54197963612084 - - type: recall - value: 95.44316474712068 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-eng_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-eng_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 98.14722083124687 - - type: f1 - value: 97.57135703555333 - - type: main_score - value: 97.57135703555333 - - type: precision - value: 97.2959439158738 - - type: recall - value: 98.14722083124687 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fas_Arab - name: MTEB NTREXBitextMining (rus_Cyrl-fas_Arab) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.64196294441662 - - type: f1 - value: 93.24653647137372 - - type: main_score - value: 93.24653647137372 - - type: precision - value: 92.60724419963279 - - type: recall - value: 94.64196294441662 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fin_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-fin_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 87.98197295943916 - - type: f1 - value: 85.23368385912201 - - type: main_score - value: 85.23368385912201 - - type: precision - value: 84.08159858835873 - - type: recall - value: 87.98197295943916 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-fra_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-fra_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.24436654982473 - - type: f1 - value: 95.07093974294774 - - type: main_score - value: 95.07093974294774 - - type: precision - value: 94.49591053246536 - - type: recall - value: 96.24436654982473 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-heb_Hebr - name: MTEB NTREXBitextMining (rus_Cyrl-heb_Hebr) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 91.08662994491738 - - type: f1 - value: 88.5161074945752 - - type: main_score - value: 88.5161074945752 - - type: precision - value: 87.36187614755467 - - type: recall - value: 91.08662994491738 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hin_Deva - name: MTEB NTREXBitextMining (rus_Cyrl-hin_Deva) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.04256384576865 - - type: f1 - value: 93.66382907694876 - - type: main_score - value: 93.66382907694876 - - type: precision - value: 93.05291270238692 - - type: recall - value: 95.04256384576865 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hrv_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-hrv_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.14271407110667 - - type: f1 - value: 93.7481221832749 - - type: main_score - value: 93.7481221832749 - - type: precision - value: 93.10930681736892 - - type: recall - value: 95.14271407110667 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-hun_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-hun_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 90.18527791687532 - - type: f1 - value: 87.61415933423946 - - type: main_score - value: 87.61415933423946 - - type: precision - value: 86.5166400394242 - - type: recall - value: 90.18527791687532 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ind_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-ind_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.69053580370556 - - type: f1 - value: 91.83608746453012 - - type: main_score - value: 91.83608746453012 - - type: precision - value: 90.97145718577868 - - type: recall - value: 93.69053580370556 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-jpn_Jpan - name: MTEB NTREXBitextMining (rus_Cyrl-jpn_Jpan) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 89.48422633950926 - - type: f1 - value: 86.91271033534429 - - type: main_score - value: 86.91271033534429 - - type: precision - value: 85.82671626487351 - - type: recall - value: 89.48422633950926 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-kor_Hang - name: MTEB NTREXBitextMining (rus_Cyrl-kor_Hang) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 88.4827240861292 - - type: f1 - value: 85.35080398375342 - - type: main_score - value: 85.35080398375342 - - type: precision - value: 83.9588549490903 - - type: recall - value: 88.4827240861292 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-lit_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-lit_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 90.33550325488233 - - type: f1 - value: 87.68831819157307 - - type: main_score - value: 87.68831819157307 - - type: precision - value: 86.51524906407231 - - type: recall - value: 90.33550325488233 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-mkd_Cyrl - name: MTEB NTREXBitextMining (rus_Cyrl-mkd_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.94391587381071 - - type: f1 - value: 94.90402270071775 - - type: main_score - value: 94.90402270071775 - - type: precision - value: 94.43915873810715 - - type: recall - value: 95.94391587381071 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-nld_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-nld_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 92.98948422633951 - - type: f1 - value: 91.04323151393756 - - type: main_score - value: 91.04323151393756 - - type: precision - value: 90.14688699716241 - - type: recall - value: 92.98948422633951 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-pol_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-pol_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.34151226840261 - - type: f1 - value: 92.8726422967785 - - type: main_score - value: 92.8726422967785 - - type: precision - value: 92.19829744616925 - - type: recall - value: 94.34151226840261 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-por_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-por_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 86.17926890335504 - - type: f1 - value: 82.7304882287356 - - type: main_score - value: 82.7304882287356 - - type: precision - value: 81.28162481817964 - - type: recall - value: 86.17926890335504 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-slk_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-slk_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 92.7391086629945 - - type: f1 - value: 90.75112669003506 - - type: main_score - value: 90.75112669003506 - - type: precision - value: 89.8564513436822 - - type: recall - value: 92.7391086629945 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-slv_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-slv_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 92.8893340010015 - - type: f1 - value: 91.05992321816058 - - type: main_score - value: 91.05992321816058 - - type: precision - value: 90.22589439715128 - - type: recall - value: 92.8893340010015 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-spa_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-spa_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.49474211316975 - - type: f1 - value: 95.4715406442998 - - type: main_score - value: 95.4715406442998 - - type: precision - value: 94.9799699549324 - - type: recall - value: 96.49474211316975 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-srp_Cyrl - name: MTEB NTREXBitextMining (rus_Cyrl-srp_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 81.07160741111667 - - type: f1 - value: 76.55687285507015 - - type: main_score - value: 76.55687285507015 - - type: precision - value: 74.71886401030116 - - type: recall - value: 81.07160741111667 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-srp_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-srp_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.14271407110667 - - type: f1 - value: 93.73302377809138 - - type: main_score - value: 93.73302377809138 - - type: precision - value: 93.06960440660991 - - type: recall - value: 95.14271407110667 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-swa_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-swa_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.79218828242364 - - type: f1 - value: 93.25988983475212 - - type: main_score - value: 93.25988983475212 - - type: precision - value: 92.53463528626273 - - type: recall - value: 94.79218828242364 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-swe_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-swe_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.04256384576865 - - type: f1 - value: 93.58704723752295 - - type: main_score - value: 93.58704723752295 - - type: precision - value: 92.91437155733601 - - type: recall - value: 95.04256384576865 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tam_Taml - name: MTEB NTREXBitextMining (rus_Cyrl-tam_Taml) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.28993490235354 - - type: f1 - value: 91.63912535469872 - - type: main_score - value: 91.63912535469872 - - type: precision - value: 90.87738750983617 - - type: recall - value: 93.28993490235354 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-tur_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-tur_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.74061091637456 - - type: f1 - value: 91.96628275746953 - - type: main_score - value: 91.96628275746953 - - type: precision - value: 91.15923885828742 - - type: recall - value: 93.74061091637456 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-ukr_Cyrl - name: MTEB NTREXBitextMining (rus_Cyrl-ukr_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.99399098647972 - - type: f1 - value: 94.89567684860624 - - type: main_score - value: 94.89567684860624 - - type: precision - value: 94.37072275079286 - - type: recall - value: 95.99399098647972 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-vie_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-vie_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 91.4371557336004 - - type: f1 - value: 88.98681355366382 - - type: main_score - value: 88.98681355366382 - - type: precision - value: 87.89183775663496 - - type: recall - value: 91.4371557336004 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-zho_Hant - name: MTEB NTREXBitextMining (rus_Cyrl-zho_Hant) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 92.7891837756635 - - type: f1 - value: 90.79047142141783 - - type: main_score - value: 90.79047142141783 - - type: precision - value: 89.86980470706058 - - type: recall - value: 92.7891837756635 - task: - type: BitextMining - - dataset: - config: rus_Cyrl-zul_Latn - name: MTEB NTREXBitextMining (rus_Cyrl-zul_Latn) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 87.43114672008012 - - type: f1 - value: 84.04618833011422 - - type: main_score - value: 84.04618833011422 - - type: precision - value: 82.52259341393041 - - type: recall - value: 87.43114672008012 - task: - type: BitextMining - - dataset: - config: slk_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (slk_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.34301452178268 - - type: f1 - value: 94.20392493502158 - - type: main_score - value: 94.20392493502158 - - type: precision - value: 93.67384409948257 - - type: recall - value: 95.34301452178268 - task: - type: BitextMining - - dataset: - config: slv_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (slv_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 92.23835753630446 - - type: f1 - value: 90.5061759305625 - - type: main_score - value: 90.5061759305625 - - type: precision - value: 89.74231188051918 - - type: recall - value: 92.23835753630446 - task: - type: BitextMining - - dataset: - config: spa_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (spa_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.54481722583876 - - type: f1 - value: 95.54665331330328 - - type: main_score - value: 95.54665331330328 - - type: precision - value: 95.06342847604739 - - type: recall - value: 96.54481722583876 - task: - type: BitextMining - - dataset: - config: srp_Cyrl-rus_Cyrl - name: MTEB NTREXBitextMining (srp_Cyrl-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 83.62543815723585 - - type: f1 - value: 80.77095672699816 - - type: main_score - value: 80.77095672699816 - - type: precision - value: 79.74674313056886 - - type: recall - value: 83.62543815723585 - task: - type: BitextMining - - dataset: - config: srp_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (srp_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 94.44166249374061 - - type: f1 - value: 93.00733206591994 - - type: main_score - value: 93.00733206591994 - - type: precision - value: 92.37203026762366 - - type: recall - value: 94.44166249374061 - task: - type: BitextMining - - dataset: - config: swa_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (swa_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 90.23535302954431 - - type: f1 - value: 87.89596482636041 - - type: main_score - value: 87.89596482636041 - - type: precision - value: 86.87060227370694 - - type: recall - value: 90.23535302954431 - task: - type: BitextMining - - dataset: - config: swe_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (swe_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 95.44316474712068 - - type: f1 - value: 94.1896177599733 - - type: main_score - value: 94.1896177599733 - - type: precision - value: 93.61542313470206 - - type: recall - value: 95.44316474712068 - task: - type: BitextMining - - dataset: - config: tam_Taml-rus_Cyrl - name: MTEB NTREXBitextMining (tam_Taml-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 89.68452679018529 - - type: f1 - value: 87.37341160650037 - - type: main_score - value: 87.37341160650037 - - type: precision - value: 86.38389402285247 - - type: recall - value: 89.68452679018529 - task: - type: BitextMining - - dataset: - config: tur_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (tur_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.89083625438157 - - type: f1 - value: 92.33892505424804 - - type: main_score - value: 92.33892505424804 - - type: precision - value: 91.63125640842216 - - type: recall - value: 93.89083625438157 - task: - type: BitextMining - - dataset: - config: ukr_Cyrl-rus_Cyrl - name: MTEB NTREXBitextMining (ukr_Cyrl-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 96.14421632448673 - - type: f1 - value: 95.11028447433054 - - type: main_score - value: 95.11028447433054 - - type: precision - value: 94.62944416624937 - - type: recall - value: 96.14421632448673 - task: - type: BitextMining - - dataset: - config: vie_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (vie_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 93.79068602904357 - - type: f1 - value: 92.14989150392256 - - type: main_score - value: 92.14989150392256 - - type: precision - value: 91.39292271740945 - - type: recall - value: 93.79068602904357 - task: - type: BitextMining - - dataset: - config: zho_Hant-rus_Cyrl - name: MTEB NTREXBitextMining (zho_Hant-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 89.13370055082625 - - type: f1 - value: 86.51514618639217 - - type: main_score - value: 86.51514618639217 - - type: precision - value: 85.383920035898 - - type: recall - value: 89.13370055082625 - task: - type: BitextMining - - dataset: - config: zul_Latn-rus_Cyrl - name: MTEB NTREXBitextMining (zul_Latn-rus_Cyrl) - revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 - split: test - type: mteb/NTREX - metrics: - - type: accuracy - value: 81.17175763645467 - - type: f1 - value: 77.72331766047338 - - type: main_score - value: 77.72331766047338 - - type: precision - value: 76.24629555848075 - - type: recall - value: 81.17175763645467 - task: - type: BitextMining - - dataset: - config: ru - name: MTEB OpusparcusPC (ru) - revision: 9e9b1f8ef51616073f47f306f7f47dd91663f86a - split: test.full - type: GEM/opusparcus - metrics: - - type: cosine_accuracy - value: 73.09136420525657 - - type: cosine_accuracy_threshold - value: 87.70400881767273 - - type: cosine_ap - value: 86.51938550599533 - - type: cosine_f1 - value: 80.84358523725834 - - type: cosine_f1_threshold - value: 86.90648078918457 - - type: cosine_precision - value: 73.24840764331209 - - type: cosine_recall - value: 90.19607843137256 - - type: dot_accuracy - value: 73.09136420525657 - - type: dot_accuracy_threshold - value: 87.7040147781372 - - type: dot_ap - value: 86.51934769946833 - - type: dot_f1 - value: 80.84358523725834 - - type: dot_f1_threshold - value: 86.90648078918457 - - type: dot_precision - value: 73.24840764331209 - - type: dot_recall - value: 90.19607843137256 - - type: euclidean_accuracy - value: 73.09136420525657 - - type: euclidean_accuracy_threshold - value: 49.590304493904114 - - type: euclidean_ap - value: 86.51934769946833 - - type: euclidean_f1 - value: 80.84358523725834 - - type: euclidean_f1_threshold - value: 51.173269748687744 - - type: euclidean_precision - value: 73.24840764331209 - - type: euclidean_recall - value: 90.19607843137256 - - type: main_score - value: 86.51976811057995 - - type: manhattan_accuracy - value: 73.40425531914893 - - type: manhattan_accuracy_threshold - value: 757.8278541564941 - - type: manhattan_ap - value: 86.51976811057995 - - type: manhattan_f1 - value: 80.92898615453328 - - type: manhattan_f1_threshold - value: 778.3821105957031 - - type: manhattan_precision - value: 74.32321575061526 - - type: manhattan_recall - value: 88.8235294117647 - - type: max_ap - value: 86.51976811057995 - - type: max_f1 - value: 80.92898615453328 - - type: max_precision - value: 74.32321575061526 - - type: max_recall - value: 90.19607843137256 - - type: similarity_accuracy - value: 73.09136420525657 - - type: similarity_accuracy_threshold - value: 87.70400881767273 - - type: similarity_ap - value: 86.51938550599533 - - type: similarity_f1 - value: 80.84358523725834 - - type: similarity_f1_threshold - value: 86.90648078918457 - - type: similarity_precision - value: 73.24840764331209 - - type: similarity_recall - value: 90.19607843137256 - task: - type: PairClassification - - dataset: - config: russian - name: MTEB PublicHealthQA (russian) - revision: main - split: test - type: xhluca/publichealth-qa - metrics: - - type: main_score - value: 79.303 - - type: map_at_1 - value: 61.538000000000004 - - type: map_at_10 - value: 74.449 - - type: map_at_100 - value: 74.687 - - type: map_at_1000 - value: 74.687 - - type: map_at_20 - value: 74.589 - - type: map_at_3 - value: 73.333 - - type: map_at_5 - value: 74.256 - - type: mrr_at_1 - value: 61.53846153846154 - - type: mrr_at_10 - value: 74.44871794871794 - - type: mrr_at_100 - value: 74.68730304304074 - - type: mrr_at_1000 - value: 74.68730304304074 - - type: mrr_at_20 - value: 74.58857808857809 - - type: mrr_at_3 - value: 73.33333333333333 - - type: mrr_at_5 - value: 74.25641025641025 - - type: nauc_map_at_1000_diff1 - value: 61.375798048778506 - - type: nauc_map_at_1000_max - value: 51.37093181241067 - - type: nauc_map_at_1000_std - value: 41.735794471409015 - - type: nauc_map_at_100_diff1 - value: 61.375798048778506 - - type: nauc_map_at_100_max - value: 51.37093181241067 - - type: nauc_map_at_100_std - value: 41.735794471409015 - - type: nauc_map_at_10_diff1 - value: 61.12796039757213 - - type: nauc_map_at_10_max - value: 51.843445267118014 - - type: nauc_map_at_10_std - value: 42.243121474939365 - - type: nauc_map_at_1_diff1 - value: 66.39100974909151 - - type: nauc_map_at_1_max - value: 44.77165601342703 - - type: nauc_map_at_1_std - value: 32.38542979413408 - - type: nauc_map_at_20_diff1 - value: 61.16611123434347 - - type: nauc_map_at_20_max - value: 51.52605092407306 - - type: nauc_map_at_20_std - value: 41.94787773313971 - - type: nauc_map_at_3_diff1 - value: 61.40157474408937 - - type: nauc_map_at_3_max - value: 51.47230077853947 - - type: nauc_map_at_3_std - value: 42.63540269440141 - - type: nauc_map_at_5_diff1 - value: 61.07631147583098 - - type: nauc_map_at_5_max - value: 52.02626939341523 - - type: nauc_map_at_5_std - value: 42.511607332150334 - - type: nauc_mrr_at_1000_diff1 - value: 61.375798048778506 - - type: nauc_mrr_at_1000_max - value: 51.37093181241067 - - type: nauc_mrr_at_1000_std - value: 41.735794471409015 - - type: nauc_mrr_at_100_diff1 - value: 61.375798048778506 - - type: nauc_mrr_at_100_max - value: 51.37093181241067 - - type: nauc_mrr_at_100_std - value: 41.735794471409015 - - type: nauc_mrr_at_10_diff1 - value: 61.12796039757213 - - type: nauc_mrr_at_10_max - value: 51.843445267118014 - - type: nauc_mrr_at_10_std - value: 42.243121474939365 - - type: nauc_mrr_at_1_diff1 - value: 66.39100974909151 - - type: nauc_mrr_at_1_max - value: 44.77165601342703 - - type: nauc_mrr_at_1_std - value: 32.38542979413408 - - type: nauc_mrr_at_20_diff1 - value: 61.16611123434347 - - type: nauc_mrr_at_20_max - value: 51.52605092407306 - - type: nauc_mrr_at_20_std - value: 41.94787773313971 - - type: nauc_mrr_at_3_diff1 - value: 61.40157474408937 - - type: nauc_mrr_at_3_max - value: 51.47230077853947 - - type: nauc_mrr_at_3_std - value: 42.63540269440141 - - type: nauc_mrr_at_5_diff1 - value: 61.07631147583098 - - type: nauc_mrr_at_5_max - value: 52.02626939341523 - - type: nauc_mrr_at_5_std - value: 42.511607332150334 - - type: nauc_ndcg_at_1000_diff1 - value: 60.54821630436157 - - type: nauc_ndcg_at_1000_max - value: 52.584328363863634 - - type: nauc_ndcg_at_1000_std - value: 43.306961101645946 - - type: nauc_ndcg_at_100_diff1 - value: 60.54821630436157 - - type: nauc_ndcg_at_100_max - value: 52.584328363863634 - - type: nauc_ndcg_at_100_std - value: 43.306961101645946 - - type: nauc_ndcg_at_10_diff1 - value: 58.800340278109886 - - type: nauc_ndcg_at_10_max - value: 55.31050771670664 - - type: nauc_ndcg_at_10_std - value: 46.40931672942848 - - type: nauc_ndcg_at_1_diff1 - value: 66.39100974909151 - - type: nauc_ndcg_at_1_max - value: 44.77165601342703 - - type: nauc_ndcg_at_1_std - value: 32.38542979413408 - - type: nauc_ndcg_at_20_diff1 - value: 58.88690479697946 - - type: nauc_ndcg_at_20_max - value: 54.19269661177923 - - type: nauc_ndcg_at_20_std - value: 45.39305589413174 - - type: nauc_ndcg_at_3_diff1 - value: 59.61866351451574 - - type: nauc_ndcg_at_3_max - value: 54.23992718744033 - - type: nauc_ndcg_at_3_std - value: 46.997379274101 - - type: nauc_ndcg_at_5_diff1 - value: 58.70739588066225 - - type: nauc_ndcg_at_5_max - value: 55.76766902539152 - - type: nauc_ndcg_at_5_std - value: 47.10553115762958 - - type: nauc_precision_at_1000_diff1 - value: 100.0 - - type: nauc_precision_at_1000_max - value: 100.0 - - type: nauc_precision_at_1000_std - value: 100.0 - - type: nauc_precision_at_100_diff1 - value: .nan - - type: nauc_precision_at_100_max - value: .nan - - type: nauc_precision_at_100_std - value: .nan - - type: nauc_precision_at_10_diff1 - value: 35.72622112397501 - - type: nauc_precision_at_10_max - value: 89.84297108673948 - - type: nauc_precision_at_10_std - value: 86.60269192422707 - - type: nauc_precision_at_1_diff1 - value: 66.39100974909151 - - type: nauc_precision_at_1_max - value: 44.77165601342703 - - type: nauc_precision_at_1_std - value: 32.38542979413408 - - type: nauc_precision_at_20_diff1 - value: 29.188449183726433 - - type: nauc_precision_at_20_max - value: 86.45729478231968 - - type: nauc_precision_at_20_std - value: 86.45729478231968 - - type: nauc_precision_at_3_diff1 - value: 50.294126629236224 - - type: nauc_precision_at_3_max - value: 68.98223127174579 - - type: nauc_precision_at_3_std - value: 70.31195520376356 - - type: nauc_precision_at_5_diff1 - value: 39.648884288124385 - - type: nauc_precision_at_5_max - value: 86.3409770687935 - - type: nauc_precision_at_5_std - value: 83.74875373878356 - - type: nauc_recall_at_1000_diff1 - value: .nan - - type: nauc_recall_at_1000_max - value: .nan - - type: nauc_recall_at_1000_std - value: .nan - - type: nauc_recall_at_100_diff1 - value: .nan - - type: nauc_recall_at_100_max - value: .nan - - type: nauc_recall_at_100_std - value: .nan - - type: nauc_recall_at_10_diff1 - value: 35.72622112397516 - - type: nauc_recall_at_10_max - value: 89.84297108673968 - - type: nauc_recall_at_10_std - value: 86.60269192422749 - - type: nauc_recall_at_1_diff1 - value: 66.39100974909151 - - type: nauc_recall_at_1_max - value: 44.77165601342703 - - type: nauc_recall_at_1_std - value: 32.38542979413408 - - type: nauc_recall_at_20_diff1 - value: 29.188449183726323 - - type: nauc_recall_at_20_max - value: 86.45729478231985 - - type: nauc_recall_at_20_std - value: 86.45729478231985 - - type: nauc_recall_at_3_diff1 - value: 50.29412662923603 - - type: nauc_recall_at_3_max - value: 68.98223127174562 - - type: nauc_recall_at_3_std - value: 70.31195520376346 - - type: nauc_recall_at_5_diff1 - value: 39.64888428812445 - - type: nauc_recall_at_5_max - value: 86.34097706879359 - - type: nauc_recall_at_5_std - value: 83.74875373878366 - - type: ndcg_at_1 - value: 61.538000000000004 - - type: ndcg_at_10 - value: 79.303 - - type: ndcg_at_100 - value: 80.557 - - type: ndcg_at_1000 - value: 80.557 - - type: ndcg_at_20 - value: 79.732 - - type: ndcg_at_3 - value: 77.033 - - type: ndcg_at_5 - value: 78.818 - - type: precision_at_1 - value: 61.538000000000004 - - type: precision_at_10 - value: 9.385 - - type: precision_at_100 - value: 1.0 - - type: precision_at_1000 - value: 0.1 - - type: precision_at_20 - value: 4.769 - - type: precision_at_3 - value: 29.231 - - type: precision_at_5 - value: 18.462 - - type: recall_at_1 - value: 61.538000000000004 - - type: recall_at_10 - value: 93.84599999999999 - - type: recall_at_100 - value: 100.0 - - type: recall_at_1000 - value: 100.0 - - type: recall_at_20 - value: 95.38499999999999 - - type: recall_at_3 - value: 87.69200000000001 - - type: recall_at_5 - value: 92.308 - task: - type: Retrieval - - dataset: - config: default - name: MTEB RUParaPhraserSTS (default) - revision: 43265056790b8f7c59e0139acb4be0a8dad2c8f4 - split: test - type: merionum/ru_paraphraser - metrics: - - type: cosine_pearson - value: 64.73554596215753 - - type: cosine_spearman - value: 70.45849652271855 - - type: euclidean_pearson - value: 68.08069844834267 - - type: euclidean_spearman - value: 70.45854872959124 - - type: main_score - value: 70.45849652271855 - - type: manhattan_pearson - value: 67.88325986519624 - - type: manhattan_spearman - value: 70.21131896834542 - - type: pearson - value: 64.73554596215753 - - type: spearman - value: 70.45849652271855 - task: - type: STS - - dataset: - config: default - name: MTEB RiaNewsRetrieval (default) - revision: 82374b0bbacda6114f39ff9c5b925fa1512ca5d7 - split: test - type: ai-forever/ria-news-retrieval - metrics: - - type: main_score - value: 70.00999999999999 - - type: map_at_1 - value: 55.97 - - type: map_at_10 - value: 65.59700000000001 - - type: map_at_100 - value: 66.057 - - type: map_at_1000 - value: 66.074 - - type: map_at_20 - value: 65.892 - - type: map_at_3 - value: 63.74999999999999 - - type: map_at_5 - value: 64.84299999999999 - - type: mrr_at_1 - value: 55.88999999999999 - - type: mrr_at_10 - value: 65.55873015872977 - - type: mrr_at_100 - value: 66.01891495129716 - - type: mrr_at_1000 - value: 66.03538391493299 - - type: mrr_at_20 - value: 65.85351193431555 - - type: mrr_at_3 - value: 63.7133333333329 - - type: mrr_at_5 - value: 64.80483333333268 - - type: nauc_map_at_1000_diff1 - value: 65.95332946436318 - - type: nauc_map_at_1000_max - value: 28.21204156197811 - - type: nauc_map_at_1000_std - value: -13.139245767083743 - - type: nauc_map_at_100_diff1 - value: 65.94763105024367 - - type: nauc_map_at_100_max - value: 28.212832170078205 - - type: nauc_map_at_100_std - value: -13.131425849370665 - - type: nauc_map_at_10_diff1 - value: 65.88455089448388 - - type: nauc_map_at_10_max - value: 28.13555838776792 - - type: nauc_map_at_10_std - value: -13.326989827081023 - - type: nauc_map_at_1_diff1 - value: 69.31275711813979 - - type: nauc_map_at_1_max - value: 26.386708520283758 - - type: nauc_map_at_1_std - value: -14.434616447245464 - - type: nauc_map_at_20_diff1 - value: 65.91227032605677 - - type: nauc_map_at_20_max - value: 28.20538655600886 - - type: nauc_map_at_20_std - value: -13.191148834410274 - - type: nauc_map_at_3_diff1 - value: 66.0051677952641 - - type: nauc_map_at_3_max - value: 28.25443420019022 - - type: nauc_map_at_3_std - value: -13.893284109029558 - - type: nauc_map_at_5_diff1 - value: 65.89784348297898 - - type: nauc_map_at_5_max - value: 28.26449765184183 - - type: nauc_map_at_5_std - value: -13.506692912805008 - - type: nauc_mrr_at_1000_diff1 - value: 66.06599513750889 - - type: nauc_mrr_at_1000_max - value: 28.191556650722287 - - type: nauc_mrr_at_1000_std - value: -13.098487982930276 - - type: nauc_mrr_at_100_diff1 - value: 66.0602307977725 - - type: nauc_mrr_at_100_max - value: 28.19235936624514 - - type: nauc_mrr_at_100_std - value: -13.09069677716269 - - type: nauc_mrr_at_10_diff1 - value: 65.99546819079403 - - type: nauc_mrr_at_10_max - value: 28.11556170120022 - - type: nauc_mrr_at_10_std - value: -13.286711073897553 - - type: nauc_mrr_at_1_diff1 - value: 69.49541040517995 - - type: nauc_mrr_at_1_max - value: 26.354622707276153 - - type: nauc_mrr_at_1_std - value: -14.358839778104695 - - type: nauc_mrr_at_20_diff1 - value: 66.02427154257936 - - type: nauc_mrr_at_20_max - value: 28.18509383563462 - - type: nauc_mrr_at_20_std - value: -13.150543398429 - - type: nauc_mrr_at_3_diff1 - value: 66.11258119082618 - - type: nauc_mrr_at_3_max - value: 28.239510722224004 - - type: nauc_mrr_at_3_std - value: -13.857249251136269 - - type: nauc_mrr_at_5_diff1 - value: 66.00633786765626 - - type: nauc_mrr_at_5_max - value: 28.244875152193032 - - type: nauc_mrr_at_5_std - value: -13.467206028704434 - - type: nauc_ndcg_at_1000_diff1 - value: 65.02876183314446 - - type: nauc_ndcg_at_1000_max - value: 29.109368390197194 - - type: nauc_ndcg_at_1000_std - value: -11.56514359821697 - - type: nauc_ndcg_at_100_diff1 - value: 64.85837726893713 - - type: nauc_ndcg_at_100_max - value: 29.19990133137256 - - type: nauc_ndcg_at_100_std - value: -11.17450348161257 - - type: nauc_ndcg_at_10_diff1 - value: 64.53842705024796 - - type: nauc_ndcg_at_10_max - value: 28.748734006088526 - - type: nauc_ndcg_at_10_std - value: -12.331395505957063 - - type: nauc_ndcg_at_1_diff1 - value: 69.31275711813979 - - type: nauc_ndcg_at_1_max - value: 26.386708520283758 - - type: nauc_ndcg_at_1_std - value: -14.434616447245464 - - type: nauc_ndcg_at_20_diff1 - value: 64.59017606740504 - - type: nauc_ndcg_at_20_max - value: 29.047332048898017 - - type: nauc_ndcg_at_20_std - value: -11.746548770195954 - - type: nauc_ndcg_at_3_diff1 - value: 64.87900935713822 - - type: nauc_ndcg_at_3_max - value: 28.953157521204403 - - type: nauc_ndcg_at_3_std - value: -13.639947228880942 - - type: nauc_ndcg_at_5_diff1 - value: 64.61466953479034 - - type: nauc_ndcg_at_5_max - value: 29.01899321868392 - - type: nauc_ndcg_at_5_std - value: -12.85356404799802 - - type: nauc_precision_at_1000_diff1 - value: 48.85481417002382 - - type: nauc_precision_at_1000_max - value: 57.129837326696375 - - type: nauc_precision_at_1000_std - value: 37.889524999906435 - - type: nauc_precision_at_100_diff1 - value: 53.374672326788264 - - type: nauc_precision_at_100_max - value: 43.819333062207974 - - type: nauc_precision_at_100_std - value: 21.387064885769362 - - type: nauc_precision_at_10_diff1 - value: 57.66571169774445 - - type: nauc_precision_at_10_max - value: 31.779694837242033 - - type: nauc_precision_at_10_std - value: -6.6248399147180255 - - type: nauc_precision_at_1_diff1 - value: 69.31275711813979 - - type: nauc_precision_at_1_max - value: 26.386708520283758 - - type: nauc_precision_at_1_std - value: -14.434616447245464 - - type: nauc_precision_at_20_diff1 - value: 55.93570036001682 - - type: nauc_precision_at_20_max - value: 34.98640173388743 - - type: nauc_precision_at_20_std - value: -0.36518465159326174 - - type: nauc_precision_at_3_diff1 - value: 60.94100093991508 - - type: nauc_precision_at_3_max - value: 31.422239034357673 - - type: nauc_precision_at_3_std - value: -12.72576556537896 - - type: nauc_precision_at_5_diff1 - value: 59.450505195434054 - - type: nauc_precision_at_5_max - value: 32.07638712418377 - - type: nauc_precision_at_5_std - value: -10.024459103498598 - - type: nauc_recall_at_1000_diff1 - value: 48.854814170024184 - - type: nauc_recall_at_1000_max - value: 57.129837326697164 - - type: nauc_recall_at_1000_std - value: 37.88952499990672 - - type: nauc_recall_at_100_diff1 - value: 53.37467232678822 - - type: nauc_recall_at_100_max - value: 43.8193330622079 - - type: nauc_recall_at_100_std - value: 21.387064885769398 - - type: nauc_recall_at_10_diff1 - value: 57.66571169774447 - - type: nauc_recall_at_10_max - value: 31.779694837242133 - - type: nauc_recall_at_10_std - value: -6.62483991471789 - - type: nauc_recall_at_1_diff1 - value: 69.31275711813979 - - type: nauc_recall_at_1_max - value: 26.386708520283758 - - type: nauc_recall_at_1_std - value: -14.434616447245464 - - type: nauc_recall_at_20_diff1 - value: 55.93570036001682 - - type: nauc_recall_at_20_max - value: 34.986401733887554 - - type: nauc_recall_at_20_std - value: -0.3651846515931506 - - type: nauc_recall_at_3_diff1 - value: 60.94100093991499 - - type: nauc_recall_at_3_max - value: 31.422239034357606 - - type: nauc_recall_at_3_std - value: -12.725765565378966 - - type: nauc_recall_at_5_diff1 - value: 59.450505195434125 - - type: nauc_recall_at_5_max - value: 32.07638712418387 - - type: nauc_recall_at_5_std - value: -10.024459103498472 - - type: ndcg_at_1 - value: 55.97 - - type: ndcg_at_10 - value: 70.00999999999999 - - type: ndcg_at_100 - value: 72.20100000000001 - - type: ndcg_at_1000 - value: 72.65599999999999 - - type: ndcg_at_20 - value: 71.068 - - type: ndcg_at_3 - value: 66.228 - - type: ndcg_at_5 - value: 68.191 - - type: precision_at_1 - value: 55.97 - - type: precision_at_10 - value: 8.373999999999999 - - type: precision_at_100 - value: 0.9390000000000001 - - type: precision_at_1000 - value: 0.097 - - type: precision_at_20 - value: 4.3950000000000005 - - type: precision_at_3 - value: 24.46 - - type: precision_at_5 - value: 15.626000000000001 - - type: recall_at_1 - value: 55.97 - - type: recall_at_10 - value: 83.74000000000001 - - type: recall_at_100 - value: 93.87 - - type: recall_at_1000 - value: 97.49 - - type: recall_at_20 - value: 87.89 - - type: recall_at_3 - value: 73.38 - - type: recall_at_5 - value: 78.13 - task: - type: Retrieval - - dataset: - config: default - name: MTEB RuBQReranking (default) - revision: 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2 - split: test - type: ai-forever/rubq-reranking - metrics: - - type: main_score - value: 71.44929565043827 - - type: map - value: 71.44929565043827 - - type: mrr - value: 77.78391820945014 - - type: nAUC_map_diff1 - value: 38.140840668080244 - - type: nAUC_map_max - value: 27.54328688105381 - - type: nAUC_map_std - value: 16.81572082284672 - - type: nAUC_mrr_diff1 - value: 44.51350415961509 - - type: nAUC_mrr_max - value: 36.491182016669754 - - type: nAUC_mrr_std - value: 22.47139593052269 - task: - type: Reranking - - dataset: - config: default - name: MTEB RuBQRetrieval (default) - revision: e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b - split: test - type: ai-forever/rubq-retrieval - metrics: - - type: main_score - value: 68.529 - - type: map_at_1 - value: 42.529 - - type: map_at_10 - value: 60.864 - - type: map_at_100 - value: 61.868 - - type: map_at_1000 - value: 61.907000000000004 - - type: map_at_20 - value: 61.596 - - type: map_at_3 - value: 55.701 - - type: map_at_5 - value: 58.78 - - type: mrr_at_1 - value: 60.57919621749409 - - type: mrr_at_10 - value: 70.55614188149649 - - type: mrr_at_100 - value: 70.88383816664494 - - type: mrr_at_1000 - value: 70.89719252668833 - - type: mrr_at_20 - value: 70.79839750105347 - - type: mrr_at_3 - value: 68.4594168636722 - - type: mrr_at_5 - value: 69.67100078802214 - - type: nauc_map_at_1000_diff1 - value: 40.67438785660885 - - type: nauc_map_at_1000_max - value: 32.79981738507424 - - type: nauc_map_at_1000_std - value: -6.873402600044831 - - type: nauc_map_at_100_diff1 - value: 40.65643664443284 - - type: nauc_map_at_100_max - value: 32.81594799919249 - - type: nauc_map_at_100_std - value: -6.8473246794498195 - - type: nauc_map_at_10_diff1 - value: 40.39048268484908 - - type: nauc_map_at_10_max - value: 32.403242161479525 - - type: nauc_map_at_10_std - value: -7.344413799841244 - - type: nauc_map_at_1_diff1 - value: 44.36306892906905 - - type: nauc_map_at_1_max - value: 25.61348630699028 - - type: nauc_map_at_1_std - value: -8.713074613333902 - - type: nauc_map_at_20_diff1 - value: 40.530326570124615 - - type: nauc_map_at_20_max - value: 32.74028319323205 - - type: nauc_map_at_20_std - value: -7.008180779820569 - - type: nauc_map_at_3_diff1 - value: 40.764924859364044 - - type: nauc_map_at_3_max - value: 29.809671682025336 - - type: nauc_map_at_3_std - value: -9.205620202725564 - - type: nauc_map_at_5_diff1 - value: 40.88599496021476 - - type: nauc_map_at_5_max - value: 32.1701894666848 - - type: nauc_map_at_5_std - value: -7.801251849010623 - - type: nauc_mrr_at_1000_diff1 - value: 48.64181373540728 - - type: nauc_mrr_at_1000_max - value: 40.136947990653546 - - type: nauc_mrr_at_1000_std - value: -7.250260497468805 - - type: nauc_mrr_at_100_diff1 - value: 48.63349902496212 - - type: nauc_mrr_at_100_max - value: 40.14510559704008 - - type: nauc_mrr_at_100_std - value: -7.228702374801103 - - type: nauc_mrr_at_10_diff1 - value: 48.58580560194813 - - type: nauc_mrr_at_10_max - value: 40.15075599433366 - - type: nauc_mrr_at_10_std - value: -7.267928771548688 - - type: nauc_mrr_at_1_diff1 - value: 51.47535097164919 - - type: nauc_mrr_at_1_max - value: 38.23579750430856 - - type: nauc_mrr_at_1_std - value: -9.187785187137633 - - type: nauc_mrr_at_20_diff1 - value: 48.58688378336222 - - type: nauc_mrr_at_20_max - value: 40.13408744088299 - - type: nauc_mrr_at_20_std - value: -7.283132775160146 - - type: nauc_mrr_at_3_diff1 - value: 48.66833005454742 - - type: nauc_mrr_at_3_max - value: 40.07987333638038 - - type: nauc_mrr_at_3_std - value: -7.738819947521418 - - type: nauc_mrr_at_5_diff1 - value: 48.76536305941537 - - type: nauc_mrr_at_5_max - value: 40.381929739522185 - - type: nauc_mrr_at_5_std - value: -7.592858318378928 - - type: nauc_ndcg_at_1000_diff1 - value: 41.67304442004693 - - type: nauc_ndcg_at_1000_max - value: 35.84126926253235 - - type: nauc_ndcg_at_1000_std - value: -4.78971011604655 - - type: nauc_ndcg_at_100_diff1 - value: 41.16918850185783 - - type: nauc_ndcg_at_100_max - value: 36.082461962326505 - - type: nauc_ndcg_at_100_std - value: -4.092442251697269 - - type: nauc_ndcg_at_10_diff1 - value: 40.300065598615205 - - type: nauc_ndcg_at_10_max - value: 34.87866296788365 - - type: nauc_ndcg_at_10_std - value: -5.866529277842453 - - type: nauc_ndcg_at_1_diff1 - value: 51.74612915209495 - - type: nauc_ndcg_at_1_max - value: 37.71907067970078 - - type: nauc_ndcg_at_1_std - value: -9.064124266098696 - - type: nauc_ndcg_at_20_diff1 - value: 40.493949850214584 - - type: nauc_ndcg_at_20_max - value: 35.69331503650286 - - type: nauc_ndcg_at_20_std - value: -4.995310342975443 - - type: nauc_ndcg_at_3_diff1 - value: 41.269443212112364 - - type: nauc_ndcg_at_3_max - value: 32.572844460953334 - - type: nauc_ndcg_at_3_std - value: -9.063015396458791 - - type: nauc_ndcg_at_5_diff1 - value: 41.37039652522888 - - type: nauc_ndcg_at_5_max - value: 34.67416011393571 - - type: nauc_ndcg_at_5_std - value: -7.106845569862319 - - type: nauc_precision_at_1000_diff1 - value: -9.571769961090155 - - type: nauc_precision_at_1000_max - value: 5.574782583417188 - - type: nauc_precision_at_1000_std - value: 7.28333847923847 - - type: nauc_precision_at_100_diff1 - value: -7.7405012003383735 - - type: nauc_precision_at_100_max - value: 9.67745355070353 - - type: nauc_precision_at_100_std - value: 9.327890294080992 - - type: nauc_precision_at_10_diff1 - value: -1.006879647532931 - - type: nauc_precision_at_10_max - value: 15.899825481231064 - - type: nauc_precision_at_10_std - value: 4.2284084852153105 - - type: nauc_precision_at_1_diff1 - value: 51.74612915209495 - - type: nauc_precision_at_1_max - value: 37.71907067970078 - - type: nauc_precision_at_1_std - value: -9.064124266098696 - - type: nauc_precision_at_20_diff1 - value: -4.982301544401409 - - type: nauc_precision_at_20_max - value: 13.241674471380568 - - type: nauc_precision_at_20_std - value: 7.052280133821539 - - type: nauc_precision_at_3_diff1 - value: 15.442614376387374 - - type: nauc_precision_at_3_max - value: 25.12695418083 - - type: nauc_precision_at_3_std - value: -3.1150066697920638 - - type: nauc_precision_at_5_diff1 - value: 8.381026072692444 - - type: nauc_precision_at_5_max - value: 22.839056540604822 - - type: nauc_precision_at_5_std - value: 1.5126905486524331 - - type: nauc_recall_at_1000_diff1 - value: -0.8869709920433502 - - type: nauc_recall_at_1000_max - value: 45.092324433377264 - - type: nauc_recall_at_1000_std - value: 62.21264093315108 - - type: nauc_recall_at_100_diff1 - value: 16.036715011075714 - - type: nauc_recall_at_100_max - value: 39.79963411771158 - - type: nauc_recall_at_100_std - value: 28.41850069503361 - - type: nauc_recall_at_10_diff1 - value: 25.189622794479998 - - type: nauc_recall_at_10_max - value: 30.82355277039427 - - type: nauc_recall_at_10_std - value: 0.0964544736531047 - - type: nauc_recall_at_1_diff1 - value: 44.36306892906905 - - type: nauc_recall_at_1_max - value: 25.61348630699028 - - type: nauc_recall_at_1_std - value: -8.713074613333902 - - type: nauc_recall_at_20_diff1 - value: 20.43424504746087 - - type: nauc_recall_at_20_max - value: 33.96010554649377 - - type: nauc_recall_at_20_std - value: 6.900984030301936 - - type: nauc_recall_at_3_diff1 - value: 33.86531858793492 - - type: nauc_recall_at_3_max - value: 27.725692256711188 - - type: nauc_recall_at_3_std - value: -8.533124289305709 - - type: nauc_recall_at_5_diff1 - value: 32.006964557701686 - - type: nauc_recall_at_5_max - value: 31.493370659289806 - - type: nauc_recall_at_5_std - value: -4.8639793547793255 - - type: ndcg_at_1 - value: 60.461 - - type: ndcg_at_10 - value: 68.529 - - type: ndcg_at_100 - value: 71.664 - - type: ndcg_at_1000 - value: 72.396 - - type: ndcg_at_20 - value: 70.344 - - type: ndcg_at_3 - value: 61.550000000000004 - - type: ndcg_at_5 - value: 64.948 - - type: precision_at_1 - value: 60.461 - - type: precision_at_10 - value: 13.28 - - type: precision_at_100 - value: 1.555 - - type: precision_at_1000 - value: 0.164 - - type: precision_at_20 - value: 7.216 - - type: precision_at_3 - value: 33.077 - - type: precision_at_5 - value: 23.014000000000003 - - type: recall_at_1 - value: 42.529 - - type: recall_at_10 - value: 81.169 - - type: recall_at_100 - value: 93.154 - - type: recall_at_1000 - value: 98.18299999999999 - - type: recall_at_20 - value: 87.132 - - type: recall_at_3 - value: 63.905 - - type: recall_at_5 - value: 71.967 - task: - type: Retrieval - - dataset: - config: default - name: MTEB RuReviewsClassification (default) - revision: f6d2c31f4dc6b88f468552750bfec05b4b41b05a - split: test - type: ai-forever/ru-reviews-classification - metrics: - - type: accuracy - value: 61.17675781250001 - - type: f1 - value: 60.354535346041374 - - type: f1_weighted - value: 60.35437313166116 - - type: main_score - value: 61.17675781250001 - task: - type: Classification - - dataset: - config: default - name: MTEB RuSTSBenchmarkSTS (default) - revision: 7cf24f325c6da6195df55bef3d86b5e0616f3018 - split: test - type: ai-forever/ru-stsbenchmark-sts - metrics: - - type: cosine_pearson - value: 78.1301041727274 - - type: cosine_spearman - value: 78.08238025421747 - - type: euclidean_pearson - value: 77.35224254583635 - - type: euclidean_spearman - value: 78.08235336582496 - - type: main_score - value: 78.08238025421747 - - type: manhattan_pearson - value: 77.24138550052075 - - type: manhattan_spearman - value: 77.98199107904142 - - type: pearson - value: 78.1301041727274 - - type: spearman - value: 78.08238025421747 - task: - type: STS - - dataset: - config: default - name: MTEB RuSciBenchGRNTIClassification (default) - revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1 - split: test - type: ai-forever/ru-scibench-grnti-classification - metrics: - - type: accuracy - value: 54.990234375 - - type: f1 - value: 53.537019057131374 - - type: f1_weighted - value: 53.552745354520766 - - type: main_score - value: 54.990234375 - task: - type: Classification - - dataset: - config: default - name: MTEB RuSciBenchGRNTIClusteringP2P (default) - revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1 - split: test - type: ai-forever/ru-scibench-grnti-classification - metrics: - - type: main_score - value: 50.775228895355106 - - type: v_measure - value: 50.775228895355106 - - type: v_measure_std - value: 0.9533571150165796 - task: - type: Clustering - - dataset: - config: default - name: MTEB RuSciBenchOECDClassification (default) - revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471 - split: test - type: ai-forever/ru-scibench-oecd-classification - metrics: - - type: accuracy - value: 41.71875 - - type: f1 - value: 39.289100975858304 - - type: f1_weighted - value: 39.29257829217775 - - type: main_score - value: 41.71875 - task: - type: Classification - - dataset: - config: default - name: MTEB RuSciBenchOECDClusteringP2P (default) - revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471 - split: test - type: ai-forever/ru-scibench-oecd-classification - metrics: - - type: main_score - value: 45.10904808834516 - - type: v_measure - value: 45.10904808834516 - - type: v_measure_std - value: 1.0572643410157534 - task: - type: Clustering - - dataset: - config: rus_Cyrl - name: MTEB SIB200Classification (rus_Cyrl) - revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b - split: test - type: mteb/sib200 - metrics: - - type: accuracy - value: 66.36363636363637 - - type: f1 - value: 64.6940336621617 - - type: f1_weighted - value: 66.43317771876966 - - type: main_score - value: 66.36363636363637 - task: - type: Classification - - dataset: - config: rus_Cyrl - name: MTEB SIB200ClusteringS2S (rus_Cyrl) - revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b - split: test - type: mteb/sib200 - metrics: - - type: main_score - value: 33.99178497314711 - - type: v_measure - value: 33.99178497314711 - - type: v_measure_std - value: 4.036337464043786 - task: - type: Clustering - - dataset: - config: ru - name: MTEB STS22.v2 (ru) - revision: d31f33a128469b20e357535c39b82fb3c3f6f2bd - split: test - type: mteb/sts22-crosslingual-sts - metrics: - - type: cosine_pearson - value: 50.724322379215934 - - type: cosine_spearman - value: 59.90449732164651 - - type: euclidean_pearson - value: 50.227545226784024 - - type: euclidean_spearman - value: 59.898906527601085 - - type: main_score - value: 59.90449732164651 - - type: manhattan_pearson - value: 50.21762139819405 - - type: manhattan_spearman - value: 59.761039813759 - - type: pearson - value: 50.724322379215934 - - type: spearman - value: 59.90449732164651 - task: - type: STS - - dataset: - config: ru - name: MTEB STSBenchmarkMultilingualSTS (ru) - revision: 29afa2569dcedaaa2fe6a3dcfebab33d28b82e8c - split: dev - type: mteb/stsb_multi_mt - metrics: - - type: cosine_pearson - value: 78.43928769569945 - - type: cosine_spearman - value: 78.23961768018884 - - type: euclidean_pearson - value: 77.4718694027985 - - type: euclidean_spearman - value: 78.23887044760475 - - type: main_score - value: 78.23961768018884 - - type: manhattan_pearson - value: 77.34517128089547 - - type: manhattan_spearman - value: 78.1146477340426 - - type: pearson - value: 78.43928769569945 - - type: spearman - value: 78.23961768018884 - task: - type: STS - - dataset: - config: default - name: MTEB SensitiveTopicsClassification (default) - revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2 - split: test - type: ai-forever/sensitive-topics-classification - metrics: - - type: accuracy - value: 22.8125 - - type: f1 - value: 17.31969589593409 - - type: lrap - value: 33.82412380642287 - - type: main_score - value: 22.8125 - task: - type: MultilabelClassification - - dataset: - config: default - name: MTEB TERRa (default) - revision: 7b58f24536063837d644aab9a023c62199b2a612 - split: dev - type: ai-forever/terra-pairclassification - metrics: - - type: cosine_accuracy - value: 57.32899022801303 - - type: cosine_accuracy_threshold - value: 85.32201051712036 - - type: cosine_ap - value: 55.14264553720072 - - type: cosine_f1 - value: 66.83544303797468 - - type: cosine_f1_threshold - value: 85.32201051712036 - - type: cosine_precision - value: 54.54545454545454 - - type: cosine_recall - value: 86.27450980392157 - - type: dot_accuracy - value: 57.32899022801303 - - type: dot_accuracy_threshold - value: 85.32201051712036 - - type: dot_ap - value: 55.14264553720072 - - type: dot_f1 - value: 66.83544303797468 - - type: dot_f1_threshold - value: 85.32201051712036 - - type: dot_precision - value: 54.54545454545454 - - type: dot_recall - value: 86.27450980392157 - - type: euclidean_accuracy - value: 57.32899022801303 - - type: euclidean_accuracy_threshold - value: 54.18117046356201 - - type: euclidean_ap - value: 55.14264553720072 - - type: euclidean_f1 - value: 66.83544303797468 - - type: euclidean_f1_threshold - value: 54.18117046356201 - - type: euclidean_precision - value: 54.54545454545454 - - type: euclidean_recall - value: 86.27450980392157 - - type: main_score - value: 55.14264553720072 - - type: manhattan_accuracy - value: 57.32899022801303 - - type: manhattan_accuracy_threshold - value: 828.8480758666992 - - type: manhattan_ap - value: 55.077974053622555 - - type: manhattan_f1 - value: 66.82352941176471 - - type: manhattan_f1_threshold - value: 885.6784820556641 - - type: manhattan_precision - value: 52.20588235294118 - - type: manhattan_recall - value: 92.81045751633987 - - type: max_ap - value: 55.14264553720072 - - type: max_f1 - value: 66.83544303797468 - - type: max_precision - value: 54.54545454545454 - - type: max_recall - value: 92.81045751633987 - - type: similarity_accuracy - value: 57.32899022801303 - - type: similarity_accuracy_threshold - value: 85.32201051712036 - - type: similarity_ap - value: 55.14264553720072 - - type: similarity_f1 - value: 66.83544303797468 - - type: similarity_f1_threshold - value: 85.32201051712036 - - type: similarity_precision - value: 54.54545454545454 - - type: similarity_recall - value: 86.27450980392157 - task: - type: PairClassification - - dataset: - config: ru - name: MTEB XNLI (ru) - revision: 09698e0180d87dc247ca447d3a1248b931ac0cdb - split: test - type: mteb/xnli - metrics: - - type: cosine_accuracy - value: 67.6923076923077 - - type: cosine_accuracy_threshold - value: 87.6681923866272 - - type: cosine_ap - value: 73.18693800863593 - - type: cosine_f1 - value: 70.40641099026904 - - type: cosine_f1_threshold - value: 85.09706258773804 - - type: cosine_precision - value: 57.74647887323944 - - type: cosine_recall - value: 90.17595307917888 - - type: dot_accuracy - value: 67.6923076923077 - - type: dot_accuracy_threshold - value: 87.66818642616272 - - type: dot_ap - value: 73.18693800863593 - - type: dot_f1 - value: 70.40641099026904 - - type: dot_f1_threshold - value: 85.09706258773804 - - type: dot_precision - value: 57.74647887323944 - - type: dot_recall - value: 90.17595307917888 - - type: euclidean_accuracy - value: 67.6923076923077 - - type: euclidean_accuracy_threshold - value: 49.662476778030396 - - type: euclidean_ap - value: 73.18693800863593 - - type: euclidean_f1 - value: 70.40641099026904 - - type: euclidean_f1_threshold - value: 54.59475517272949 - - type: euclidean_precision - value: 57.74647887323944 - - type: euclidean_recall - value: 90.17595307917888 - - type: main_score - value: 73.18693800863593 - - type: manhattan_accuracy - value: 67.54578754578755 - - type: manhattan_accuracy_threshold - value: 777.1001815795898 - - type: manhattan_ap - value: 72.98861474758783 - - type: manhattan_f1 - value: 70.6842435655995 - - type: manhattan_f1_threshold - value: 810.3782653808594 - - type: manhattan_precision - value: 61.80021953896817 - - type: manhattan_recall - value: 82.55131964809385 - - type: max_ap - value: 73.18693800863593 - - type: max_f1 - value: 70.6842435655995 - - type: max_precision - value: 61.80021953896817 - - type: max_recall - value: 90.17595307917888 - - type: similarity_accuracy - value: 67.6923076923077 - - type: similarity_accuracy_threshold - value: 87.6681923866272 - - type: similarity_ap - value: 73.18693800863593 - - type: similarity_f1 - value: 70.40641099026904 - - type: similarity_f1_threshold - value: 85.09706258773804 - - type: similarity_precision - value: 57.74647887323944 - - type: similarity_recall - value: 90.17595307917888 - task: - type: PairClassification - - dataset: - config: russian - name: MTEB XNLIV2 (russian) - revision: 5b7d477a8c62cdd18e2fed7e015497c20b4371ad - split: test - type: mteb/xnli2.0-multi-pair - metrics: - - type: cosine_accuracy - value: 68.35164835164835 - - type: cosine_accuracy_threshold - value: 88.48621845245361 - - type: cosine_ap - value: 73.10205506215699 - - type: cosine_f1 - value: 71.28712871287128 - - type: cosine_f1_threshold - value: 87.00399398803711 - - type: cosine_precision - value: 61.67023554603854 - - type: cosine_recall - value: 84.4574780058651 - - type: dot_accuracy - value: 68.35164835164835 - - type: dot_accuracy_threshold - value: 88.48622441291809 - - type: dot_ap - value: 73.10191110714706 - - type: dot_f1 - value: 71.28712871287128 - - type: dot_f1_threshold - value: 87.00399398803711 - - type: dot_precision - value: 61.67023554603854 - - type: dot_recall - value: 84.4574780058651 - - type: euclidean_accuracy - value: 68.35164835164835 - - type: euclidean_accuracy_threshold - value: 47.98704385757446 - - type: euclidean_ap - value: 73.10205506215699 - - type: euclidean_f1 - value: 71.28712871287128 - - type: euclidean_f1_threshold - value: 50.982362031936646 - - type: euclidean_precision - value: 61.67023554603854 - - type: euclidean_recall - value: 84.4574780058651 - - type: main_score - value: 73.10205506215699 - - type: manhattan_accuracy - value: 67.91208791208791 - - type: manhattan_accuracy_threshold - value: 746.1360931396484 - - type: manhattan_ap - value: 72.8954736175069 - - type: manhattan_f1 - value: 71.1297071129707 - - type: manhattan_f1_threshold - value: 808.0789566040039 - - type: manhattan_precision - value: 60.04036326942482 - - type: manhattan_recall - value: 87.2434017595308 - - type: max_ap - value: 73.10205506215699 - - type: max_f1 - value: 71.28712871287128 - - type: max_precision - value: 61.67023554603854 - - type: max_recall - value: 87.2434017595308 - - type: similarity_accuracy - value: 68.35164835164835 - - type: similarity_accuracy_threshold - value: 88.48621845245361 - - type: similarity_ap - value: 73.10205506215699 - - type: similarity_f1 - value: 71.28712871287128 - - type: similarity_f1_threshold - value: 87.00399398803711 - - type: similarity_precision - value: 61.67023554603854 - - type: similarity_recall - value: 84.4574780058651 - task: - type: PairClassification - - dataset: - config: ru - name: MTEB XQuADRetrieval (ru) - revision: 51adfef1c1287aab1d2d91b5bead9bcfb9c68583 - split: validation - type: google/xquad - metrics: - - type: main_score - value: 95.705 - - type: map_at_1 - value: 90.802 - - type: map_at_10 - value: 94.427 - - type: map_at_100 - value: 94.451 - - type: map_at_1000 - value: 94.451 - - type: map_at_20 - value: 94.446 - - type: map_at_3 - value: 94.121 - - type: map_at_5 - value: 94.34 - - type: mrr_at_1 - value: 90.80168776371308 - - type: mrr_at_10 - value: 94.42659567343111 - - type: mrr_at_100 - value: 94.45099347521871 - - type: mrr_at_1000 - value: 94.45099347521871 - - type: mrr_at_20 - value: 94.44574530017569 - - type: mrr_at_3 - value: 94.12095639943743 - - type: mrr_at_5 - value: 94.34036568213786 - - type: nauc_map_at_1000_diff1 - value: 87.40573202946949 - - type: nauc_map_at_1000_max - value: 65.56220344468791 - - type: nauc_map_at_1000_std - value: 8.865583291735863 - - type: nauc_map_at_100_diff1 - value: 87.40573202946949 - - type: nauc_map_at_100_max - value: 65.56220344468791 - - type: nauc_map_at_100_std - value: 8.865583291735863 - - type: nauc_map_at_10_diff1 - value: 87.43657080570291 - - type: nauc_map_at_10_max - value: 65.71295628534446 - - type: nauc_map_at_10_std - value: 9.055399339099655 - - type: nauc_map_at_1_diff1 - value: 88.08395824560428 - - type: nauc_map_at_1_max - value: 62.92813192908893 - - type: nauc_map_at_1_std - value: 6.738987385482432 - - type: nauc_map_at_20_diff1 - value: 87.40979818966589 - - type: nauc_map_at_20_max - value: 65.59474346926105 - - type: nauc_map_at_20_std - value: 8.944420599300914 - - type: nauc_map_at_3_diff1 - value: 86.97771892161035 - - type: nauc_map_at_3_max - value: 66.14330030122467 - - type: nauc_map_at_3_std - value: 8.62516327793521 - - type: nauc_map_at_5_diff1 - value: 87.30273362211798 - - type: nauc_map_at_5_max - value: 66.1522476584607 - - type: nauc_map_at_5_std - value: 9.780940862679724 - - type: nauc_mrr_at_1000_diff1 - value: 87.40573202946949 - - type: nauc_mrr_at_1000_max - value: 65.56220344468791 - - type: nauc_mrr_at_1000_std - value: 8.865583291735863 - - type: nauc_mrr_at_100_diff1 - value: 87.40573202946949 - - type: nauc_mrr_at_100_max - value: 65.56220344468791 - - type: nauc_mrr_at_100_std - value: 8.865583291735863 - - type: nauc_mrr_at_10_diff1 - value: 87.43657080570291 - - type: nauc_mrr_at_10_max - value: 65.71295628534446 - - type: nauc_mrr_at_10_std - value: 9.055399339099655 - - type: nauc_mrr_at_1_diff1 - value: 88.08395824560428 - - type: nauc_mrr_at_1_max - value: 62.92813192908893 - - type: nauc_mrr_at_1_std - value: 6.738987385482432 - - type: nauc_mrr_at_20_diff1 - value: 87.40979818966589 - - type: nauc_mrr_at_20_max - value: 65.59474346926105 - - type: nauc_mrr_at_20_std - value: 8.944420599300914 - - type: nauc_mrr_at_3_diff1 - value: 86.97771892161035 - - type: nauc_mrr_at_3_max - value: 66.14330030122467 - - type: nauc_mrr_at_3_std - value: 8.62516327793521 - - type: nauc_mrr_at_5_diff1 - value: 87.30273362211798 - - type: nauc_mrr_at_5_max - value: 66.1522476584607 - - type: nauc_mrr_at_5_std - value: 9.780940862679724 - - type: nauc_ndcg_at_1000_diff1 - value: 87.37823158814116 - - type: nauc_ndcg_at_1000_max - value: 66.00874244792789 - - type: nauc_ndcg_at_1000_std - value: 9.479929342875067 - - type: nauc_ndcg_at_100_diff1 - value: 87.37823158814116 - - type: nauc_ndcg_at_100_max - value: 66.00874244792789 - - type: nauc_ndcg_at_100_std - value: 9.479929342875067 - - type: nauc_ndcg_at_10_diff1 - value: 87.54508467181488 - - type: nauc_ndcg_at_10_max - value: 66.88756470312894 - - type: nauc_ndcg_at_10_std - value: 10.812624405397022 - - type: nauc_ndcg_at_1_diff1 - value: 88.08395824560428 - - type: nauc_ndcg_at_1_max - value: 62.92813192908893 - - type: nauc_ndcg_at_1_std - value: 6.738987385482432 - - type: nauc_ndcg_at_20_diff1 - value: 87.42097894104597 - - type: nauc_ndcg_at_20_max - value: 66.37031898778943 - - type: nauc_ndcg_at_20_std - value: 10.34862538094813 - - type: nauc_ndcg_at_3_diff1 - value: 86.50039907157999 - - type: nauc_ndcg_at_3_max - value: 67.97798288917929 - - type: nauc_ndcg_at_3_std - value: 10.162410286746852 - - type: nauc_ndcg_at_5_diff1 - value: 87.13322094568531 - - type: nauc_ndcg_at_5_max - value: 68.08576118683821 - - type: nauc_ndcg_at_5_std - value: 12.639637379592855 - - type: nauc_precision_at_1000_diff1 - value: 100.0 - - type: nauc_precision_at_1000_max - value: 100.0 - - type: nauc_precision_at_1000_std - value: 100.0 - - type: nauc_precision_at_100_diff1 - value: 100.0 - - type: nauc_precision_at_100_max - value: 100.0 - - type: nauc_precision_at_100_std - value: 100.0 - - type: nauc_precision_at_10_diff1 - value: 93.46711505595813 - - type: nauc_precision_at_10_max - value: 100.0 - - type: nauc_precision_at_10_std - value: 65.42573557179935 - - type: nauc_precision_at_1_diff1 - value: 88.08395824560428 - - type: nauc_precision_at_1_max - value: 62.92813192908893 - - type: nauc_precision_at_1_std - value: 6.738987385482432 - - type: nauc_precision_at_20_diff1 - value: 91.28948674127133 - - type: nauc_precision_at_20_max - value: 100.0 - - type: nauc_precision_at_20_std - value: 90.74278258632364 - - type: nauc_precision_at_3_diff1 - value: 82.64606115071832 - - type: nauc_precision_at_3_max - value: 83.26201582412921 - - type: nauc_precision_at_3_std - value: 23.334013491433762 - - type: nauc_precision_at_5_diff1 - value: 85.0867539350284 - - type: nauc_precision_at_5_max - value: 96.57011448655484 - - type: nauc_precision_at_5_std - value: 56.46869543426768 - - type: nauc_recall_at_1000_diff1 - value: .nan - - type: nauc_recall_at_1000_max - value: .nan - - type: nauc_recall_at_1000_std - value: .nan - - type: nauc_recall_at_100_diff1 - value: .nan - - type: nauc_recall_at_100_max - value: .nan - - type: nauc_recall_at_100_std - value: .nan - - type: nauc_recall_at_10_diff1 - value: 93.46711505595623 - - type: nauc_recall_at_10_max - value: 100.0 - - type: nauc_recall_at_10_std - value: 65.42573557180279 - - type: nauc_recall_at_1_diff1 - value: 88.08395824560428 - - type: nauc_recall_at_1_max - value: 62.92813192908893 - - type: nauc_recall_at_1_std - value: 6.738987385482432 - - type: nauc_recall_at_20_diff1 - value: 91.28948674127474 - - type: nauc_recall_at_20_max - value: 100.0 - - type: nauc_recall_at_20_std - value: 90.74278258632704 - - type: nauc_recall_at_3_diff1 - value: 82.64606115071967 - - type: nauc_recall_at_3_max - value: 83.26201582413023 - - type: nauc_recall_at_3_std - value: 23.334013491434007 - - type: nauc_recall_at_5_diff1 - value: 85.08675393502854 - - type: nauc_recall_at_5_max - value: 96.57011448655487 - - type: nauc_recall_at_5_std - value: 56.46869543426658 - - type: ndcg_at_1 - value: 90.802 - - type: ndcg_at_10 - value: 95.705 - - type: ndcg_at_100 - value: 95.816 - - type: ndcg_at_1000 - value: 95.816 - - type: ndcg_at_20 - value: 95.771 - - type: ndcg_at_3 - value: 95.11699999999999 - - type: ndcg_at_5 - value: 95.506 - - type: precision_at_1 - value: 90.802 - - type: precision_at_10 - value: 9.949 - - type: precision_at_100 - value: 1.0 - - type: precision_at_1000 - value: 0.1 - - type: precision_at_20 - value: 4.987 - - type: precision_at_3 - value: 32.658 - - type: precision_at_5 - value: 19.781000000000002 - - type: recall_at_1 - value: 90.802 - - type: recall_at_10 - value: 99.494 - - type: recall_at_100 - value: 100.0 - - type: recall_at_1000 - value: 100.0 - - type: recall_at_20 - value: 99.747 - - type: recall_at_3 - value: 97.975 - - type: recall_at_5 - value: 98.90299999999999 - task: - type: Retrieval -tags: -- mteb -- Sentence Transformers -- sentence-similarity -- sentence-transformers ---- - - -## Multilingual-E5-small - -[Multilingual E5 Text Embeddings: A Technical Report](https://arxiv.org/pdf/2402.05672). -Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024 - -This model has 12 layers and the embedding size is 384. - -## Usage - -Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset. - -```python -import torch.nn.functional as F - -from torch import Tensor -from transformers import AutoTokenizer, AutoModel - - -def average_pool(last_hidden_states: Tensor, - attention_mask: Tensor) -> Tensor: - last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0) - return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None] - - -# Each input text should start with "query: " or "passage: ", even for non-English texts. -# For tasks other than retrieval, you can simply use the "query: " prefix. -input_texts = ['query: how much protein should a female eat', - 'query: 南瓜的家常做法', - "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.", - "passage: 1.清炒南瓜丝 原料:嫩南瓜半个 调料:葱、盐、白糖、鸡精 做法: 1、南瓜用刀薄薄的削去表面一层皮,用勺子刮去瓤 2、擦成细丝(没有擦菜板就用刀慢慢切成细丝) 3、锅烧热放油,入葱花煸出香味 4、入南瓜丝快速翻炒一分钟左右,放盐、一点白糖和鸡精调味出锅 2.香葱炒南瓜 原料:南瓜1只 调料:香葱、蒜末、橄榄油、盐 做法: 1、将南瓜去皮,切成片 2、油锅8成热后,将蒜末放入爆香 3、爆香后,将南瓜片放入,翻炒 4、在翻炒的同时,可以不时地往锅里加水,但不要太多 5、放入盐,炒匀 6、南瓜差不多软和绵了之后,就可以关火 7、撒入香葱,即可出锅"] - -tokenizer = AutoTokenizer.from_pretrained('intfloat/multilingual-e5-small') -model = AutoModel.from_pretrained('intfloat/multilingual-e5-small') - -# Tokenize the input texts -batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt') - -outputs = model(**batch_dict) -embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask']) - -# normalize embeddings -embeddings = F.normalize(embeddings, p=2, dim=1) -scores = (embeddings[:2] @ embeddings[2:].T) * 100 -print(scores.tolist()) -``` - -## Supported Languages - -This model is initialized from [microsoft/Multilingual-MiniLM-L12-H384](https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384) -and continually trained on a mixture of multilingual datasets. -It supports 100 languages from xlm-roberta, -but low-resource languages may see performance degradation. - -## Training Details - -**Initialization**: [microsoft/Multilingual-MiniLM-L12-H384](https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384) - -**First stage**: contrastive pre-training with weak supervision - -| Dataset | Weak supervision | # of text pairs | -|--------------------------------------------------------------------------------------------------------|---------------------------------------|-----------------| -| Filtered [mC4](https://huggingface.co/datasets/mc4) | (title, page content) | 1B | -| [CC News](https://huggingface.co/datasets/intfloat/multilingual_cc_news) | (title, news content) | 400M | -| [NLLB](https://huggingface.co/datasets/allenai/nllb) | translation pairs | 2.4B | -| [Wikipedia](https://huggingface.co/datasets/intfloat/wikipedia) | (hierarchical section title, passage) | 150M | -| Filtered [Reddit](https://www.reddit.com/) | (comment, response) | 800M | -| [S2ORC](https://github.com/allenai/s2orc) | (title, abstract) and citation pairs | 100M | -| [Stackexchange](https://stackexchange.com/) | (question, answer) | 50M | -| [xP3](https://huggingface.co/datasets/bigscience/xP3) | (input prompt, response) | 80M | -| [Miscellaneous unsupervised SBERT data](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | - | 10M | - -**Second stage**: supervised fine-tuning - -| Dataset | Language | # of text pairs | -|----------------------------------------------------------------------------------------|--------------|-----------------| -| [MS MARCO](https://microsoft.github.io/msmarco/) | English | 500k | -| [NQ](https://github.com/facebookresearch/DPR) | English | 70k | -| [Trivia QA](https://github.com/facebookresearch/DPR) | English | 60k | -| [NLI from SimCSE](https://github.com/princeton-nlp/SimCSE) | English | <300k | -| [ELI5](https://huggingface.co/datasets/eli5) | English | 500k | -| [DuReader Retrieval](https://github.com/baidu/DuReader/tree/master/DuReader-Retrieval) | Chinese | 86k | -| [KILT Fever](https://huggingface.co/datasets/kilt_tasks) | English | 70k | -| [KILT HotpotQA](https://huggingface.co/datasets/kilt_tasks) | English | 70k | -| [SQuAD](https://huggingface.co/datasets/squad) | English | 87k | -| [Quora](https://huggingface.co/datasets/quora) | English | 150k | -| [Mr. TyDi](https://huggingface.co/datasets/castorini/mr-tydi) | 11 languages | 50k | -| [MIRACL](https://huggingface.co/datasets/miracl/miracl) | 16 languages | 40k | - -For all labeled datasets, we only use its training set for fine-tuning. - -For other training details, please refer to our paper at [https://arxiv.org/pdf/2402.05672](https://arxiv.org/pdf/2402.05672). - -## Benchmark Results on [Mr. TyDi](https://arxiv.org/abs/2108.08787) - -| Model | Avg MRR@10 | | ar | bn | en | fi | id | ja | ko | ru | sw | te | th | -|-----------------------|------------|-------|------| --- | --- | --- | --- | --- | --- | --- |------| --- | --- | -| BM25 | 33.3 | | 36.7 | 41.3 | 15.1 | 28.8 | 38.2 | 21.7 | 28.1 | 32.9 | 39.6 | 42.4 | 41.7 | -| mDPR | 16.7 | | 26.0 | 25.8 | 16.2 | 11.3 | 14.6 | 18.1 | 21.9 | 18.5 | 7.3 | 10.6 | 13.5 | -| BM25 + mDPR | 41.7 | | 49.1 | 53.5 | 28.4 | 36.5 | 45.5 | 35.5 | 36.2 | 42.7 | 40.5 | 42.0 | 49.2 | -| | | -| multilingual-e5-small | 64.4 | | 71.5 | 66.3 | 54.5 | 57.7 | 63.2 | 55.4 | 54.3 | 60.8 | 65.4 | 89.1 | 70.1 | -| multilingual-e5-base | 65.9 | | 72.3 | 65.0 | 58.5 | 60.8 | 64.9 | 56.6 | 55.8 | 62.7 | 69.0 | 86.6 | 72.7 | -| multilingual-e5-large | **70.5** | | 77.5 | 73.2 | 60.8 | 66.8 | 68.5 | 62.5 | 61.6 | 65.8 | 72.7 | 90.2 | 76.2 | - -## MTEB Benchmark Evaluation - -Check out [unilm/e5](https://github.com/microsoft/unilm/tree/master/e5) to reproduce evaluation results -on the [BEIR](https://arxiv.org/abs/2104.08663) and [MTEB benchmark](https://arxiv.org/abs/2210.07316). - -## Support for Sentence Transformers - -Below is an example for usage with sentence_transformers. -```python -from sentence_transformers import SentenceTransformer -model = SentenceTransformer('intfloat/multilingual-e5-small') -input_texts = [ - 'query: how much protein should a female eat', - 'query: 南瓜的家常做法', - "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 i s 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or traini ng for a marathon. Check out the chart below to see how much protein you should be eating each day.", - "passage: 1.清炒南瓜丝 原料:嫩南瓜半个 调料:葱、盐、白糖、鸡精 做法: 1、南瓜用刀薄薄的削去表面一层皮 ,用勺子刮去瓤 2、擦成细丝(没有擦菜板就用刀慢慢切成细丝) 3、锅烧热放油,入葱花煸出香味 4、入南瓜丝快速翻炒一分钟左右, 放盐、一点白糖和鸡精调味出锅 2.香葱炒南瓜 原料:南瓜1只 调料:香葱、蒜末、橄榄油、盐 做法: 1、将南瓜去皮,切成片 2、油 锅8成热后,将蒜末放入爆香 3、爆香后,将南瓜片放入,翻炒 4、在翻炒的同时,可以不时地往锅里加水,但不要太多 5、放入盐,炒匀 6、南瓜差不多软和绵了之后,就可以关火 7、撒入香葱,即可出锅" -] -embeddings = model.encode(input_texts, normalize_embeddings=True) -``` - -Package requirements - -`pip install sentence_transformers~=2.2.2` - -Contributors: [michaelfeil](https://huggingface.co/michaelfeil) - -## FAQ - -**1. Do I need to add the prefix "query: " and "passage: " to input texts?** - -Yes, this is how the model is trained, otherwise you will see a performance degradation. - -Here are some rules of thumb: -- Use "query: " and "passage: " correspondingly for asymmetric tasks such as passage retrieval in open QA, ad-hoc information retrieval. - -- Use "query: " prefix for symmetric tasks such as semantic similarity, bitext mining, paraphrase retrieval. - -- Use "query: " prefix if you want to use embeddings as features, such as linear probing classification, clustering. - -**2. Why are my reproduced results slightly different from reported in the model card?** - -Different versions of `transformers` and `pytorch` could cause negligible but non-zero performance differences. - -**3. Why does the cosine similarity scores distribute around 0.7 to 1.0?** - -This is a known and expected behavior as we use a low temperature 0.01 for InfoNCE contrastive loss. - -For text embedding tasks like text retrieval or semantic similarity, -what matters is the relative order of the scores instead of the absolute values, -so this should not be an issue. - -## Citation - -If you find our paper or models helpful, please consider cite as follows: - -``` -@article{wang2024multilingual, - title={Multilingual E5 Text Embeddings: A Technical Report}, - author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu}, - journal={arXiv preprint arXiv:2402.05672}, - year={2024} -} -``` - -## Limitations - -Long texts will be truncated to at most 512 tokens. -