--- tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer widget: - text: 이건 GFK 전자 예시입니다 metrics: - accuracy pipeline_tag: text-classification library_name: setfit inference: true base_model: mini1013/master_item_top_el_flat_test model-index: - name: SetFit with mini1013/master_item_top_el_flat_test results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.9448118733836876 name: Accuracy --- # SetFit with mini1013/master_item_top_el_flat_test This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [mini1013/master_item_top_el_flat_test](https://huggingface.co/mini1013/master_item_top_el_flat_test) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [mini1013/master_item_top_el_flat_test](https://huggingface.co/mini1013/master_item_top_el_flat_test) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 256 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 176 | | | 45 | | | 23 | | | 17 | | | 201 | | | 131 | | | 225 | | | 247 | | | 148 | | | 180 | | | 76 | | | 194 | | | 42 | | | 74 | | | 87 | | | 248 | | | 29 | | | 228 | | | 133 | | | 236 | | | 213 | | | 134 | | | 186 | | | 26 | | | 51 | | | 85 | | | 124 | | | 2 | | | 222 | | | 14 | | | 156 | | | 33 | | | 246 | | | 229 | | | 243 | | | 15 | | | 19 | | | 205 | | | 63 | | | 78 | | | 101 | | | 211 | | | 118 | | | 12 | | | 254 | | | 115 | | | 197 | | | 28 | | | 16 | | | 184 | | | 234 | | | 90 | | | 146 | | | 142 | | | 235 | | | 207 | | | 32 | | | 203 | | | 224 | | | 249 | | | 18 | | | 117 | | | 175 | | | 86 | | | 170 | | | 237 | | | 231 | | | 174 | | | 255 | | | 218 | | | 80 | | | 173 | | | 132 | | | 217 | | | 60 | | | 21 | | | 10 | | | 212 | | | 199 | | | 160 | | | 57 | | | 25 | | | 82 | | | 168 | | | 153 | | | 152 | | | 69 | | | 55 | | | 171 | | | 145 | | | 214 | | | 226 | | | 154 | | | 151 | | | 240 | | | 35 | | | 202 | | | 204 | | | 41 | | | 169 | | | 5 | | | 61 | | | 215 | | | 206 | | | 147 | | | 6 | | | 113 | | | 70 | | | 92 | | | 233 | | | 27 | | | 149 | | | 97 | | | 20 | | | 163 | | | 150 | | | 50 | | | 137 | | | 36 | | | 244 | | | 91 | | | 38 | | | 239 | | | 4 | | | 77 | | | 158 | | | 155 | | | 89 | | | 0 | | | 111 | | | 164 | | | 129 | | | 48 | | | 125 | | | 200 | | | 110 | | | 230 | | | 116 | | | 127 | | | 227 | | | 130 | | | 62 | | | 95 | | | 84 | | | 11 | | | 120 | | | 177 | | | 3 | | | 188 | | | 238 | | | 167 | | | 143 | | | 104 | | | 182 | | | 102 | | | 37 | | | 93 | | | 79 | | | 157 | | | 68 | | | 245 | | | 43 | | | 219 | | | 139 | | | 65 | | | 208 | | | 183 | | | 126 | | | 252 | | | 73 | | | 123 | | | 221 | | | 39 | | | 13 | | | 7 | | | 83 | | | 114 | | | 121 | | | 40 | | | 54 | | | 107 | | | 198 | | | 179 | | | 88 | | | 144 | | | 251 | | | 58 | | | 119 | | | 22 | | | 53 | | | 31 | | | 178 | | | 220 | | | 96 | | | 94 | | | 108 | | | 44 | | | 195 | | | 72 | | | 241 | | | 100 | | | 190 | | | 161 | | | 166 | | | 140 | | | 185 | | | 196 | | | 136 | | | 138 | | | 216 | | | 122 | | | 165 | | | 162 | | | 135 | | | 81 | | | 250 | | | 49 | | | 98 | | | 141 | | | 103 | | | 209 | | | 181 | | | 34 | | | 223 | | | 8 | | | 30 | | | 71 | | | 66 | | | 210 | | | 189 | | | 191 | | | 56 | | | 159 | | | 24 | | | 172 | | | 75 | | | 192 | | | 105 | | | 106 | | | 242 | | | 193 | | | 128 | | | 232 | | | 47 | | | 9 | | | 46 | | | 64 | | | 1 | | | 67 | | | 109 | | | 253 | | | 59 | | | 187 | | | 112 | | | 99 | | | 52 | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.9448 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("mini1013/master_item_top_el_flat_test") # Run inference preds = model("I loved the spiderman movie!") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 7 | 24.2839 | 98 | | Label | Training Sample Count | |:------|:----------------------| | 0 | 250 | | 1 | 21 | | 2 | 250 | | 3 | 250 | | 4 | 250 | | 5 | 86 | | 6 | 193 | | 7 | 250 | | 8 | 163 | | 9 | 20 | | 10 | 209 | | 11 | 104 | | 12 | 181 | | 13 | 250 | | 14 | 250 | | 15 | 250 | | 16 | 250 | | 17 | 221 | | 18 | 249 | | 19 | 250 | | 20 | 250 | | 21 | 234 | | 22 | 250 | | 23 | 125 | | 24 | 55 | | 25 | 148 | | 26 | 203 | | 27 | 250 | | 28 | 20 | | 29 | 250 | | 30 | 20 | | 31 | 34 | | 32 | 34 | | 33 | 250 | | 34 | 65 | | 35 | 168 | | 36 | 228 | | 37 | 166 | | 38 | 250 | | 39 | 250 | | 40 | 235 | | 41 | 220 | | 42 | 250 | | 43 | 61 | | 44 | 209 | | 45 | 250 | | 46 | 20 | | 47 | 20 | | 48 | 42 | | 49 | 20 | | 50 | 250 | | 51 | 95 | | 52 | 20 | | 53 | 20 | | 54 | 25 | | 55 | 250 | | 56 | 20 | | 57 | 20 | | 58 | 250 | | 59 | 20 | | 60 | 250 | | 61 | 20 | | 62 | 20 | | 63 | 42 | | 64 | 20 | | 65 | 249 | | 66 | 20 | | 67 | 20 | | 68 | 82 | | 69 | 139 | | 70 | 250 | | 71 | 20 | | 72 | 20 | | 73 | 20 | | 74 | 250 | | 75 | 20 | | 76 | 250 | | 77 | 21 | | 78 | 88 | | 79 | 20 | | 80 | 175 | | 81 | 44 | | 82 | 67 | | 83 | 46 | | 84 | 250 | | 85 | 117 | | 86 | 250 | | 87 | 250 | | 88 | 41 | | 89 | 163 | | 90 | 250 | | 91 | 250 | | 92 | 250 | | 93 | 63 | | 94 | 28 | | 95 | 250 | | 96 | 54 | | 97 | 39 | | 98 | 29 | | 99 | 20 | | 100 | 61 | | 101 | 250 | | 102 | 208 | | 103 | 20 | | 104 | 47 | | 105 | 93 | | 106 | 42 | | 107 | 30 | | 108 | 39 | | 109 | 20 | | 110 | 238 | | 111 | 58 | | 112 | 22 | | 113 | 212 | | 114 | 20 | | 115 | 250 | | 116 | 250 | | 117 | 242 | | 118 | 250 | | 119 | 250 | | 120 | 38 | | 121 | 30 | | 122 | 94 | | 123 | 104 | | 124 | 20 | | 125 | 121 | | 126 | 53 | | 127 | 22 | | 128 | 30 | | 129 | 138 | | 130 | 216 | | 131 | 237 | | 132 | 20 | | 133 | 250 | | 134 | 250 | | 135 | 39 | | 136 | 61 | | 137 | 250 | | 138 | 37 | | 139 | 84 | | 140 | 20 | | 141 | 108 | | 142 | 61 | | 143 | 250 | | 144 | 101 | | 145 | 129 | | 146 | 203 | | 147 | 75 | | 148 | 250 | | 149 | 90 | | 150 | 250 | | 151 | 250 | | 152 | 250 | | 153 | 158 | | 154 | 250 | | 155 | 250 | | 156 | 250 | | 157 | 43 | | 158 | 20 | | 159 | 20 | | 160 | 250 | | 161 | 183 | | 162 | 29 | | 163 | 250 | | 164 | 250 | | 165 | 58 | | 166 | 30 | | 167 | 250 | | 168 | 250 | | 169 | 250 | | 170 | 242 | | 171 | 157 | | 172 | 20 | | 173 | 195 | | 174 | 250 | | 175 | 250 | | 176 | 250 | | 177 | 250 | | 178 | 185 | | 179 | 36 | | 180 | 20 | | 181 | 24 | | 182 | 100 | | 183 | 230 | | 184 | 164 | | 185 | 20 | | 186 | 250 | | 187 | 20 | | 188 | 250 | | 189 | 20 | | 190 | 20 | | 191 | 23 | | 192 | 20 | | 193 | 20 | | 194 | 250 | | 195 | 250 | | 196 | 118 | | 197 | 96 | | 198 | 238 | | 199 | 250 | | 200 | 250 | | 201 | 250 | | 202 | 250 | | 203 | 198 | | 204 | 51 | | 205 | 250 | | 206 | 250 | | 207 | 250 | | 208 | 250 | | 209 | 29 | | 210 | 20 | | 211 | 172 | | 212 | 119 | | 213 | 250 | | 214 | 250 | | 215 | 20 | | 216 | 61 | | 217 | 235 | | 218 | 146 | | 219 | 249 | | 220 | 103 | | 221 | 62 | | 222 | 250 | | 223 | 20 | | 224 | 250 | | 225 | 250 | | 226 | 250 | | 227 | 250 | | 228 | 250 | | 229 | 250 | | 230 | 250 | | 231 | 171 | | 232 | 23 | | 233 | 95 | | 234 | 250 | | 235 | 250 | | 236 | 125 | | 237 | 185 | | 238 | 195 | | 239 | 250 | | 240 | 212 | | 241 | 68 | | 242 | 24 | | 243 | 250 | | 244 | 137 | | 245 | 250 | | 246 | 250 | | 247 | 162 | | 248 | 250 | | 249 | 126 | | 250 | 20 | | 251 | 20 | | 252 | 42 | | 253 | 20 | | 254 | 250 | | 255 | 250 | ### Training Hyperparameters - batch_size: (64, 64) - num_epochs: (50, 50) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 20 - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 0.01 - loss: BatchAllTripletLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: True - warmup_proportion: 0.1 - l2_weight: 0.01 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: False ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:-------:|:-----:|:-------------:|:---------------:| | 0.0017 | 1 | 0.1239 | - | | 0.0858 | 50 | 0.0515 | - | | 0.1715 | 100 | 0.0811 | - | | 0.2573 | 150 | 0.0935 | - | | 0.3431 | 200 | 0.0939 | - | | 0.4288 | 250 | 0.0745 | - | | 0.5146 | 300 | 0.075 | - | | 0.6003 | 350 | 0.0637 | - | | 0.6861 | 400 | 0.0899 | - | | 0.7719 | 450 | 0.0949 | - | | 0.8576 | 500 | 0.0797 | - | | 0.9434 | 550 | 0.0418 | - | | 1.0292 | 600 | 0.064 | - | | 1.1149 | 650 | 0.072 | - | | 1.2007 | 700 | 0.0628 | - | | 1.2864 | 750 | 0.0845 | - | | 1.3722 | 800 | 0.0868 | - | | 1.4580 | 850 | 0.0938 | - | | 1.5437 | 900 | 0.0763 | - | | 1.6295 | 950 | 0.0784 | - | | 1.7153 | 1000 | 0.0598 | - | | 1.8010 | 1050 | 0.0758 | - | | 1.8868 | 1100 | 0.0753 | - | | 1.9726 | 1150 | 0.0773 | - | | 2.0583 | 1200 | 0.0857 | - | | 2.1441 | 1250 | 0.0961 | - | | 2.2298 | 1300 | 0.0855 | - | | 2.3156 | 1350 | 0.0818 | - | | 2.4014 | 1400 | 0.0572 | - | | 2.4871 | 1450 | 0.0762 | - | | 2.5729 | 1500 | 0.088 | - | | 2.6587 | 1550 | 0.0816 | - | | 2.7444 | 1600 | 0.0595 | - | | 2.8302 | 1650 | 0.1117 | - | | 2.9160 | 1700 | 0.0848 | - | | 3.0017 | 1750 | 0.0873 | - | | 3.0875 | 1800 | 0.0932 | - | | 3.1732 | 1850 | 0.1059 | - | | 3.2590 | 1900 | 0.0669 | - | | 3.3448 | 1950 | 0.0934 | - | | 3.4305 | 2000 | 0.0757 | - | | 3.5163 | 2050 | 0.0876 | - | | 3.6021 | 2100 | 0.0843 | - | | 3.6878 | 2150 | 0.0681 | - | | 3.7736 | 2200 | 0.0903 | - | | 3.8593 | 2250 | 0.0894 | - | | 3.9451 | 2300 | 0.0824 | - | | 4.0309 | 2350 | 0.0916 | - | | 4.1166 | 2400 | 0.0753 | - | | 4.2024 | 2450 | 0.0623 | - | | 4.2882 | 2500 | 0.0777 | - | | 4.3739 | 2550 | 0.0768 | - | | 4.4597 | 2600 | 0.0802 | - | | 4.5455 | 2650 | 0.0717 | - | | 4.6312 | 2700 | 0.0782 | - | | 4.7170 | 2750 | 0.0954 | - | | 4.8027 | 2800 | 0.0838 | - | | 4.8885 | 2850 | 0.0779 | - | | 4.9743 | 2900 | 0.078 | - | | 5.0600 | 2950 | 0.0865 | - | | 5.1458 | 3000 | 0.0842 | - | | 5.2316 | 3050 | 0.076 | - | | 5.3173 | 3100 | 0.0996 | - | | 5.4031 | 3150 | 0.0708 | - | | 5.4889 | 3200 | 0.088 | - | | 5.5746 | 3250 | 0.0962 | - | | 5.6604 | 3300 | 0.0846 | - | | 5.7461 | 3350 | 0.1029 | - | | 5.8319 | 3400 | 0.0804 | - | | 5.9177 | 3450 | 0.086 | - | | 6.0034 | 3500 | 0.1041 | - | | 6.0892 | 3550 | 0.0843 | - | | 6.1750 | 3600 | 0.0673 | - | | 6.2607 | 3650 | 0.0623 | - | | 6.3465 | 3700 | 0.0926 | - | | 6.4322 | 3750 | 0.082 | - | | 6.5180 | 3800 | 0.0775 | - | | 6.6038 | 3850 | 0.0809 | - | | 6.6895 | 3900 | 0.1028 | - | | 6.7753 | 3950 | 0.1015 | - | | 6.8611 | 4000 | 0.0833 | - | | 6.9468 | 4050 | 0.0947 | - | | 7.0326 | 4100 | 0.1037 | - | | 7.1184 | 4150 | 0.1038 | - | | 7.2041 | 4200 | 0.0797 | - | | 7.2899 | 4250 | 0.0826 | - | | 7.3756 | 4300 | 0.0835 | - | | 7.4614 | 4350 | 0.0778 | - | | 7.5472 | 4400 | 0.0922 | - | | 7.6329 | 4450 | 0.0614 | - | | 7.7187 | 4500 | 0.084 | - | | 7.8045 | 4550 | 0.0855 | - | | 7.8902 | 4600 | 0.0837 | - | | 7.9760 | 4650 | 0.0808 | - | | 8.0617 | 4700 | 0.0735 | - | | 8.1475 | 4750 | 0.0756 | - | | 8.2333 | 4800 | 0.0732 | - | | 8.3190 | 4850 | 0.0663 | - | | 8.4048 | 4900 | 0.0848 | - | | 8.4906 | 4950 | 0.1016 | - | | 8.5763 | 5000 | 0.097 | - | | 8.6621 | 5050 | 0.0846 | - | | 8.7479 | 5100 | 0.0791 | - | | 8.8336 | 5150 | 0.0894 | - | | 8.9194 | 5200 | 0.0832 | - | | 9.0051 | 5250 | 0.0777 | - | | 9.0909 | 5300 | 0.069 | - | | 9.1767 | 5350 | 0.0647 | - | | 9.2624 | 5400 | 0.0518 | - | | 9.3482 | 5450 | 0.0657 | - | | 9.4340 | 5500 | 0.0872 | - | | 9.5197 | 5550 | 0.0846 | - | | 9.6055 | 5600 | 0.082 | - | | 9.6913 | 5650 | 0.0687 | - | | 9.7770 | 5700 | 0.0884 | - | | 9.8628 | 5750 | 0.0818 | - | | 9.9485 | 5800 | 0.0798 | - | | 10.0343 | 5850 | 0.0709 | - | | 10.1201 | 5900 | 0.1066 | - | | 10.2058 | 5950 | 0.062 | - | | 10.2916 | 6000 | 0.0783 | - | | 10.3774 | 6050 | 0.067 | - | | 10.4631 | 6100 | 0.0783 | - | | 10.5489 | 6150 | 0.0822 | - | | 10.6346 | 6200 | 0.0827 | - | | 10.7204 | 6250 | 0.0587 | - | | 10.8062 | 6300 | 0.0706 | - | | 10.8919 | 6350 | 0.0707 | - | | 10.9777 | 6400 | 0.0855 | - | | 11.0635 | 6450 | 0.0545 | - | | 11.1492 | 6500 | 0.0824 | - | | 11.2350 | 6550 | 0.0693 | - | | 11.3208 | 6600 | 0.0821 | - | | 11.4065 | 6650 | 0.0807 | - | | 11.4923 | 6700 | 0.0905 | - | | 11.5780 | 6750 | 0.0524 | - | | 11.6638 | 6800 | 0.0727 | - | | 11.7496 | 6850 | 0.0693 | - | | 11.8353 | 6900 | 0.0703 | - | | 11.9211 | 6950 | 0.0639 | - | | 12.0069 | 7000 | 0.0781 | - | | 12.0926 | 7050 | 0.0625 | - | | 12.1784 | 7100 | 0.0887 | - | | 12.2642 | 7150 | 0.0672 | - | | 12.3499 | 7200 | 0.0595 | - | | 12.4357 | 7250 | 0.0755 | - | | 12.5214 | 7300 | 0.0832 | - | | 12.6072 | 7350 | 0.0705 | - | | 12.6930 | 7400 | 0.0699 | - | | 12.7787 | 7450 | 0.1 | - | | 12.8645 | 7500 | 0.0595 | - | | 12.9503 | 7550 | 0.0548 | - | | 13.0360 | 7600 | 0.084 | - | | 13.1218 | 7650 | 0.0531 | - | | 13.2075 | 7700 | 0.0745 | - | | 13.2933 | 7750 | 0.0697 | - | | 13.3791 | 7800 | 0.0933 | - | | 13.4648 | 7850 | 0.0574 | - | | 13.5506 | 7900 | 0.1014 | - | | 13.6364 | 7950 | 0.0791 | - | | 13.7221 | 8000 | 0.0986 | - | | 13.8079 | 8050 | 0.0709 | - | | 13.8937 | 8100 | 0.0678 | - | | 13.9794 | 8150 | 0.0708 | - | | 14.0652 | 8200 | 0.0513 | - | | 14.1509 | 8250 | 0.0617 | - | | 14.2367 | 8300 | 0.0619 | - | | 14.3225 | 8350 | 0.0773 | - | | 14.4082 | 8400 | 0.0476 | - | | 14.4940 | 8450 | 0.0595 | - | | 14.5798 | 8500 | 0.0828 | - | | 14.6655 | 8550 | 0.0735 | - | | 14.7513 | 8600 | 0.065 | - | | 14.8370 | 8650 | 0.0424 | - | | 14.9228 | 8700 | 0.0649 | - | | 15.0086 | 8750 | 0.0772 | - | | 15.0943 | 8800 | 0.0689 | - | | 15.1801 | 8850 | 0.0582 | - | | 15.2659 | 8900 | 0.0543 | - | | 15.3516 | 8950 | 0.0527 | - | | 15.4374 | 9000 | 0.0552 | - | | 15.5232 | 9050 | 0.0749 | - | | 15.6089 | 9100 | 0.0514 | - | | 15.6947 | 9150 | 0.0582 | - | | 15.7804 | 9200 | 0.055 | - | | 15.8662 | 9250 | 0.0616 | - | | 15.9520 | 9300 | 0.0528 | - | | 16.0377 | 9350 | 0.0606 | - | | 16.1235 | 9400 | 0.0648 | - | | 16.2093 | 9450 | 0.0752 | - | | 16.2950 | 9500 | 0.0519 | - | | 16.3808 | 9550 | 0.0518 | - | | 16.4666 | 9600 | 0.0774 | - | | 16.5523 | 9650 | 0.0816 | - | | 16.6381 | 9700 | 0.0534 | - | | 16.7238 | 9750 | 0.0664 | - | | 16.8096 | 9800 | 0.0595 | - | | 16.8954 | 9850 | 0.0534 | - | | 16.9811 | 9900 | 0.0589 | - | | 17.0669 | 9950 | 0.0529 | - | | 17.1527 | 10000 | 0.0576 | - | | 17.2384 | 10050 | 0.0595 | - | | 17.3242 | 10100 | 0.0307 | - | | 17.4099 | 10150 | 0.0458 | - | | 17.4957 | 10200 | 0.0581 | - | | 17.5815 | 10250 | 0.0529 | - | | 17.6672 | 10300 | 0.0601 | - | | 17.7530 | 10350 | 0.0542 | - | | 17.8388 | 10400 | 0.0573 | - | | 17.9245 | 10450 | 0.0717 | - | | 18.0103 | 10500 | 0.0848 | - | | 18.0961 | 10550 | 0.0737 | - | | 18.1818 | 10600 | 0.0376 | - | | 18.2676 | 10650 | 0.0498 | - | | 18.3533 | 10700 | 0.068 | - | | 18.4391 | 10750 | 0.0587 | - | | 18.5249 | 10800 | 0.0637 | - | | 18.6106 | 10850 | 0.0731 | - | | 18.6964 | 10900 | 0.0572 | - | | 18.7822 | 10950 | 0.0559 | - | | 18.8679 | 11000 | 0.0386 | - | | 18.9537 | 11050 | 0.0498 | - | | 19.0395 | 11100 | 0.0699 | - | | 19.1252 | 11150 | 0.045 | - | | 19.2110 | 11200 | 0.0531 | - | | 19.2967 | 11250 | 0.0609 | - | | 19.3825 | 11300 | 0.0632 | - | | 19.4683 | 11350 | 0.0602 | - | | 19.5540 | 11400 | 0.0609 | - | | 19.6398 | 11450 | 0.0503 | - | | 19.7256 | 11500 | 0.054 | - | | 19.8113 | 11550 | 0.0444 | - | | 19.8971 | 11600 | 0.0531 | - | | 19.9828 | 11650 | 0.0534 | - | | 20.0686 | 11700 | 0.0515 | - | | 20.1544 | 11750 | 0.0554 | - | | 20.2401 | 11800 | 0.053 | - | | 20.3259 | 11850 | 0.0706 | - | | 20.4117 | 11900 | 0.0674 | - | | 20.4974 | 11950 | 0.0474 | - | | 20.5832 | 12000 | 0.064 | - | | 20.6690 | 12050 | 0.0429 | - | | 20.7547 | 12100 | 0.0351 | - | | 20.8405 | 12150 | 0.0337 | - | | 20.9262 | 12200 | 0.0827 | - | | 21.0120 | 12250 | 0.0533 | - | | 21.0978 | 12300 | 0.0237 | - | | 21.1835 | 12350 | 0.0541 | - | | 21.2693 | 12400 | 0.0502 | - | | 21.3551 | 12450 | 0.0615 | - | | 21.4408 | 12500 | 0.0505 | - | | 21.5266 | 12550 | 0.048 | - | | 21.6123 | 12600 | 0.0595 | - | | 21.6981 | 12650 | 0.0568 | - | | 21.7839 | 12700 | 0.0503 | - | | 21.8696 | 12750 | 0.0428 | - | | 21.9554 | 12800 | 0.0466 | - | | 22.0412 | 12850 | 0.037 | - | | 22.1269 | 12900 | 0.05 | - | | 22.2127 | 12950 | 0.0552 | - | | 22.2985 | 13000 | 0.0379 | - | | 22.3842 | 13050 | 0.0391 | - | | 22.4700 | 13100 | 0.0482 | - | | 22.5557 | 13150 | 0.0491 | - | | 22.6415 | 13200 | 0.0471 | - | | 22.7273 | 13250 | 0.0307 | - | | 22.8130 | 13300 | 0.0368 | - | | 22.8988 | 13350 | 0.0364 | - | | 22.9846 | 13400 | 0.031 | - | | 23.0703 | 13450 | 0.0568 | - | | 23.1561 | 13500 | 0.0579 | - | | 23.2419 | 13550 | 0.0266 | - | | 23.3276 | 13600 | 0.0476 | - | | 23.4134 | 13650 | 0.05 | - | | 23.4991 | 13700 | 0.037 | - | | 23.5849 | 13750 | 0.0712 | - | | 23.6707 | 13800 | 0.0223 | - | | 23.7564 | 13850 | 0.045 | - | | 23.8422 | 13900 | 0.0462 | - | | 23.9280 | 13950 | 0.0352 | - | | 24.0137 | 14000 | 0.0343 | - | | 24.0995 | 14050 | 0.049 | - | | 24.1852 | 14100 | 0.039 | - | | 24.2710 | 14150 | 0.0453 | - | | 24.3568 | 14200 | 0.0531 | - | | 24.4425 | 14250 | 0.055 | - | | 24.5283 | 14300 | 0.0242 | - | | 24.6141 | 14350 | 0.0426 | - | | 24.6998 | 14400 | 0.0373 | - | | 24.7856 | 14450 | 0.0457 | - | | 24.8714 | 14500 | 0.0762 | - | | 24.9571 | 14550 | 0.0489 | - | | 25.0429 | 14600 | 0.0335 | - | | 25.1286 | 14650 | 0.0475 | - | | 25.2144 | 14700 | 0.0341 | - | | 25.3002 | 14750 | 0.0264 | - | | 25.3859 | 14800 | 0.0453 | - | | 25.4717 | 14850 | 0.0503 | - | | 25.5575 | 14900 | 0.0492 | - | | 25.6432 | 14950 | 0.044 | - | | 25.7290 | 15000 | 0.0348 | - | | 25.8148 | 15050 | 0.059 | - | | 25.9005 | 15100 | 0.0528 | - | | 25.9863 | 15150 | 0.0516 | - | | 26.0720 | 15200 | 0.0529 | - | | 26.1578 | 15250 | 0.0333 | - | | 26.2436 | 15300 | 0.0544 | - | | 26.3293 | 15350 | 0.0587 | - | | 26.4151 | 15400 | 0.0239 | - | | 26.5009 | 15450 | 0.0313 | - | | 26.5866 | 15500 | 0.0356 | - | | 26.6724 | 15550 | 0.0359 | - | | 26.7581 | 15600 | 0.0472 | - | | 26.8439 | 15650 | 0.041 | - | | 26.9297 | 15700 | 0.0445 | - | | 27.0154 | 15750 | 0.0271 | - | | 27.1012 | 15800 | 0.0366 | - | | 27.1870 | 15850 | 0.018 | - | | 27.2727 | 15900 | 0.0381 | - | | 27.3585 | 15950 | 0.0478 | - | | 27.4443 | 16000 | 0.0325 | - | | 27.5300 | 16050 | 0.0463 | - | | 27.6158 | 16100 | 0.038 | - | | 27.7015 | 16150 | 0.0522 | - | | 27.7873 | 16200 | 0.0539 | - | | 27.8731 | 16250 | 0.0294 | - | | 27.9588 | 16300 | 0.0481 | - | | 28.0446 | 16350 | 0.0335 | - | | 28.1304 | 16400 | 0.0402 | - | | 28.2161 | 16450 | 0.0425 | - | | 28.3019 | 16500 | 0.0516 | - | | 28.3877 | 16550 | 0.0288 | - | | 28.4734 | 16600 | 0.0337 | - | | 28.5592 | 16650 | 0.0225 | - | | 28.6449 | 16700 | 0.0514 | - | | 28.7307 | 16750 | 0.0503 | - | | 28.8165 | 16800 | 0.0366 | - | | 28.9022 | 16850 | 0.027 | - | | 28.9880 | 16900 | 0.0312 | - | | 29.0738 | 16950 | 0.0294 | - | | 29.1595 | 17000 | 0.0469 | - | | 29.2453 | 17050 | 0.0276 | - | | 29.3310 | 17100 | 0.0374 | - | | 29.4168 | 17150 | 0.0251 | - | | 29.5026 | 17200 | 0.0631 | - | | 29.5883 | 17250 | 0.0384 | - | | 29.6741 | 17300 | 0.0258 | - | | 29.7599 | 17350 | 0.0327 | - | | 29.8456 | 17400 | 0.0265 | - | | 29.9314 | 17450 | 0.043 | - | | 30.0172 | 17500 | 0.0245 | - | | 30.1029 | 17550 | 0.0363 | - | | 30.1887 | 17600 | 0.0329 | - | | 30.2744 | 17650 | 0.0294 | - | | 30.3602 | 17700 | 0.0475 | - | | 30.4460 | 17750 | 0.0388 | - | | 30.5317 | 17800 | 0.0486 | - | | 30.6175 | 17850 | 0.0236 | - | | 30.7033 | 17900 | 0.0339 | - | | 30.7890 | 17950 | 0.0335 | - | | 30.8748 | 18000 | 0.0396 | - | | 30.9605 | 18050 | 0.0334 | - | | 31.0463 | 18100 | 0.0169 | - | | 31.1321 | 18150 | 0.0102 | - | | 31.2178 | 18200 | 0.0318 | - | | 31.3036 | 18250 | 0.025 | - | | 31.3894 | 18300 | 0.0413 | - | | 31.4751 | 18350 | 0.0322 | - | | 31.5609 | 18400 | 0.0383 | - | | 31.6467 | 18450 | 0.0431 | - | | 31.7324 | 18500 | 0.0298 | - | | 31.8182 | 18550 | 0.0398 | - | | 31.9039 | 18600 | 0.0338 | - | | 31.9897 | 18650 | 0.0206 | - | | 32.0755 | 18700 | 0.0412 | - | | 32.1612 | 18750 | 0.018 | - | | 32.2470 | 18800 | 0.0398 | - | | 32.3328 | 18850 | 0.0233 | - | | 32.4185 | 18900 | 0.0219 | - | | 32.5043 | 18950 | 0.0312 | - | | 32.5901 | 19000 | 0.025 | - | | 32.6758 | 19050 | 0.027 | - | | 32.7616 | 19100 | 0.0312 | - | | 32.8473 | 19150 | 0.0266 | - | | 32.9331 | 19200 | 0.0397 | - | | 33.0189 | 19250 | 0.0236 | - | | 33.1046 | 19300 | 0.0296 | - | | 33.1904 | 19350 | 0.0546 | - | | 33.2762 | 19400 | 0.0291 | - | | 33.3619 | 19450 | 0.0124 | - | | 33.4477 | 19500 | 0.0408 | - | | 33.5334 | 19550 | 0.031 | - | | 33.6192 | 19600 | 0.0437 | - | | 33.7050 | 19650 | 0.0184 | - | | 33.7907 | 19700 | 0.0283 | - | | 33.8765 | 19750 | 0.0088 | - | | 33.9623 | 19800 | 0.0169 | - | | 34.0480 | 19850 | 0.0391 | - | | 34.1338 | 19900 | 0.0381 | - | | 34.2196 | 19950 | 0.0414 | - | | 34.3053 | 20000 | 0.0609 | - | | 34.3911 | 20050 | 0.0339 | - | | 34.4768 | 20100 | 0.0193 | - | | 34.5626 | 20150 | 0.0317 | - | | 34.6484 | 20200 | 0.0178 | - | | 34.7341 | 20250 | 0.0216 | - | | 34.8199 | 20300 | 0.0385 | - | | 34.9057 | 20350 | 0.0143 | - | | 34.9914 | 20400 | 0.022 | - | | 35.0772 | 20450 | 0.0176 | - | | 35.1630 | 20500 | 0.0195 | - | | 35.2487 | 20550 | 0.0091 | - | | 35.3345 | 20600 | 0.0262 | - | | 35.4202 | 20650 | 0.038 | - | | 35.5060 | 20700 | 0.0259 | - | | 35.5918 | 20750 | 0.0288 | - | | 35.6775 | 20800 | 0.0365 | - | | 35.7633 | 20850 | 0.0282 | - | | 35.8491 | 20900 | 0.0256 | - | | 35.9348 | 20950 | 0.0208 | - | | 36.0206 | 21000 | 0.0283 | - | | 36.1063 | 21050 | 0.0239 | - | | 36.1921 | 21100 | 0.0231 | - | | 36.2779 | 21150 | 0.0286 | - | | 36.3636 | 21200 | 0.0262 | - | | 36.4494 | 21250 | 0.0347 | - | | 36.5352 | 21300 | 0.0064 | - | | 36.6209 | 21350 | 0.0329 | - | | 36.7067 | 21400 | 0.0135 | - | | 36.7925 | 21450 | 0.0179 | - | | 36.8782 | 21500 | 0.0124 | - | | 36.9640 | 21550 | 0.0283 | - | | 37.0497 | 21600 | 0.0324 | - | | 37.1355 | 21650 | 0.027 | - | | 37.2213 | 21700 | 0.0202 | - | | 37.3070 | 21750 | 0.0237 | - | | 37.3928 | 21800 | 0.033 | - | | 37.4786 | 21850 | 0.0224 | - | | 37.5643 | 21900 | 0.0231 | - | | 37.6501 | 21950 | 0.0317 | - | | 37.7358 | 22000 | 0.0339 | - | | 37.8216 | 22050 | 0.0278 | - | | 37.9074 | 22100 | 0.0159 | - | | 37.9931 | 22150 | 0.0144 | - | | 38.0789 | 22200 | 0.0276 | - | | 38.1647 | 22250 | 0.0171 | - | | 38.2504 | 22300 | 0.0365 | - | | 38.3362 | 22350 | 0.0096 | - | | 38.4220 | 22400 | 0.0149 | - | | 38.5077 | 22450 | 0.0115 | - | | 38.5935 | 22500 | 0.028 | - | | 38.6792 | 22550 | 0.0171 | - | | 38.7650 | 22600 | 0.0114 | - | | 38.8508 | 22650 | 0.0132 | - | | 38.9365 | 22700 | 0.0367 | - | | 39.0223 | 22750 | 0.0301 | - | | 39.1081 | 22800 | 0.0209 | - | | 39.1938 | 22850 | 0.0308 | - | | 39.2796 | 22900 | 0.0067 | - | | 39.3654 | 22950 | 0.0156 | - | | 39.4511 | 23000 | 0.017 | - | | 39.5369 | 23050 | 0.0081 | - | | 39.6226 | 23100 | 0.032 | - | | 39.7084 | 23150 | 0.0084 | - | | 39.7942 | 23200 | 0.0063 | - | | 39.8799 | 23250 | 0.0133 | - | | 39.9657 | 23300 | 0.0304 | - | | 40.0515 | 23350 | 0.0195 | - | | 40.1372 | 23400 | 0.022 | - | | 40.2230 | 23450 | 0.0161 | - | | 40.3087 | 23500 | 0.0244 | - | | 40.3945 | 23550 | 0.0305 | - | | 40.4803 | 23600 | 0.0142 | - | | 40.5660 | 23650 | 0.0297 | - | | 40.6518 | 23700 | 0.0241 | - | | 40.7376 | 23750 | 0.0256 | - | | 40.8233 | 23800 | 0.031 | - | | 40.9091 | 23850 | 0.0311 | - | | 40.9949 | 23900 | 0.0101 | - | | 41.0806 | 23950 | 0.0194 | - | | 41.1664 | 24000 | 0.0068 | - | | 41.2521 | 24050 | 0.0211 | - | | 41.3379 | 24100 | 0.0185 | - | | 41.4237 | 24150 | 0.0136 | - | | 41.5094 | 24200 | 0.0221 | - | | 41.5952 | 24250 | 0.0229 | - | | 41.6810 | 24300 | 0.0169 | - | | 41.7667 | 24350 | 0.0157 | - | | 41.8525 | 24400 | 0.0192 | - | | 41.9383 | 24450 | 0.0176 | - | | 42.0240 | 24500 | 0.0171 | - | | 42.1098 | 24550 | 0.0073 | - | | 42.1955 | 24600 | 0.0096 | - | | 42.2813 | 24650 | 0.0225 | - | | 42.3671 | 24700 | 0.0356 | - | | 42.4528 | 24750 | 0.0157 | - | | 42.5386 | 24800 | 0.0253 | - | | 42.6244 | 24850 | 0.0298 | - | | 42.7101 | 24900 | 0.0187 | - | | 42.7959 | 24950 | 0.0104 | - | | 42.8816 | 25000 | 0.0074 | - | | 42.9674 | 25050 | 0.0149 | - | | 43.0532 | 25100 | 0.0062 | - | | 43.1389 | 25150 | 0.0203 | - | | 43.2247 | 25200 | 0.0199 | - | | 43.3105 | 25250 | 0.0257 | - | | 43.3962 | 25300 | 0.0203 | - | | 43.4820 | 25350 | 0.011 | - | | 43.5678 | 25400 | 0.0193 | - | | 43.6535 | 25450 | 0.0271 | - | | 43.7393 | 25500 | 0.0128 | - | | 43.8250 | 25550 | 0.0079 | - | | 43.9108 | 25600 | 0.0132 | - | | 43.9966 | 25650 | 0.0128 | - | | 44.0823 | 25700 | 0.0069 | - | | 44.1681 | 25750 | 0.012 | - | | 44.2539 | 25800 | 0.0282 | - | | 44.3396 | 25850 | 0.0173 | - | | 44.4254 | 25900 | 0.0319 | - | | 44.5111 | 25950 | 0.0319 | - | | 44.5969 | 26000 | 0.0295 | - | | 44.6827 | 26050 | 0.0275 | - | | 44.7684 | 26100 | 0.0199 | - | | 44.8542 | 26150 | 0.0155 | - | | 44.9400 | 26200 | 0.0135 | - | | 45.0257 | 26250 | 0.0196 | - | | 45.1115 | 26300 | 0.0077 | - | | 45.1973 | 26350 | 0.0093 | - | | 45.2830 | 26400 | 0.03 | - | | 45.3688 | 26450 | 0.0203 | - | | 45.4545 | 26500 | 0.0247 | - | | 45.5403 | 26550 | 0.0166 | - | | 45.6261 | 26600 | 0.0288 | - | | 45.7118 | 26650 | 0.0178 | - | | 45.7976 | 26700 | 0.0126 | - | | 45.8834 | 26750 | 0.0182 | - | | 45.9691 | 26800 | 0.0122 | - | | 46.0549 | 26850 | 0.0111 | - | | 46.1407 | 26900 | 0.0101 | - | | 46.2264 | 26950 | 0.0097 | - | | 46.3122 | 27000 | 0.0103 | - | | 46.3979 | 27050 | 0.0084 | - | | 46.4837 | 27100 | 0.0217 | - | | 46.5695 | 27150 | 0.011 | - | | 46.6552 | 27200 | 0.0182 | - | | 46.7410 | 27250 | 0.0222 | - | | 46.8268 | 27300 | 0.0286 | - | | 46.9125 | 27350 | 0.0097 | - | | 46.9983 | 27400 | 0.0145 | - | | 47.0840 | 27450 | 0.0075 | - | | 47.1698 | 27500 | 0.0062 | - | | 47.2556 | 27550 | 0.0162 | - | | 47.3413 | 27600 | 0.0131 | - | | 47.4271 | 27650 | 0.0197 | - | | 47.5129 | 27700 | 0.0259 | - | | 47.5986 | 27750 | 0.0226 | - | | 47.6844 | 27800 | 0.0211 | - | | 47.7702 | 27850 | 0.0055 | - | | 47.8559 | 27900 | 0.0209 | - | | 47.9417 | 27950 | 0.0289 | - | | 48.0274 | 28000 | 0.0218 | - | | 48.1132 | 28050 | 0.0139 | - | | 48.1990 | 28100 | 0.0144 | - | | 48.2847 | 28150 | 0.0052 | - | | 48.3705 | 28200 | 0.0145 | - | | 48.4563 | 28250 | 0.013 | - | | 48.5420 | 28300 | 0.0186 | - | | 48.6278 | 28350 | 0.0174 | - | | 48.7136 | 28400 | 0.0147 | - | | 48.7993 | 28450 | 0.0169 | - | | 48.8851 | 28500 | 0.0141 | - | | 48.9708 | 28550 | 0.0052 | - | | 49.0566 | 28600 | 0.0086 | - | | 49.1424 | 28650 | 0.0143 | - | | 49.2281 | 28700 | 0.003 | - | | 49.3139 | 28750 | 0.0118 | - | | 49.3997 | 28800 | 0.0055 | - | | 49.4854 | 28850 | 0.0165 | - | | 49.5712 | 28900 | 0.0177 | - | | 49.6569 | 28950 | 0.0051 | - | | 49.7427 | 29000 | 0.032 | - | | 49.8285 | 29050 | 0.0236 | - | | 49.9142 | 29100 | 0.0077 | - | | 50.0 | 29150 | 0.0124 | - | ### Framework Versions - Python: 3.11.11 - SetFit: 1.1.3 - Sentence Transformers: 5.0.0 - Transformers: 4.55.0 - PyTorch: 2.8.0.dev20250319+cu128 - Datasets: 3.1.0 - Tokenizers: 0.21.4 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```