loganh274 commited on
Commit
f08b957
·
verified ·
1 Parent(s): 2777e79

Push model using huggingface_hub.

Browse files
Files changed (3) hide show
  1. README.md +94 -153
  2. model.safetensors +1 -1
  3. model_head.pkl +1 -1
README.md CHANGED
@@ -1,189 +1,130 @@
1
  ---
2
- language:
3
- - en
4
- license: apache-2.0
5
- library_name: setfit
6
  tags:
7
  - setfit
8
  - sentence-transformers
9
  - text-classification
10
- - sentiment-analysis
11
- - few-shot-learning
12
- pipeline_tag: text-classification
13
  metrics:
14
  - accuracy
15
- - f1
16
- - precision
17
- - recall
18
- model-index:
19
- - name: SetFit Sentiment Analysis
20
- results:
21
- - task:
22
- type: text-classification
23
- name: Sentiment Analysis
24
- metrics:
25
- - name: Accuracy
26
- type: accuracy
27
- value: 0.9
28
- - name: F1 (Weighted)
29
- type: f1
30
- value: 0.8984430773904458
31
- - name: Precision (Weighted)
32
- type: precision
33
- value: 0.9060606060606061
34
- - name: Recall (Weighted)
35
- type: recall
36
- value: 0.9
37
  ---
38
 
39
- # SetFit Sentiment Analysis Model
40
-
41
- This is a [SetFit](https://github.com/huggingface/setfit) model fine-tuned for sentiment classification on customer feedback data.
42
-
43
- ## Model Description
44
-
45
- | Property | Value |
46
- |----------|-------|
47
- | **Base Model** | [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) |
48
- | **Total Parameters** | 109,482,240 |
49
- | **Trainable Parameters** | 109,482,240 |
50
- | **Body Parameters** | 109,482,240 |
51
- | **Head Parameters** | 0 |
52
- | **Model Size** | 417.64 MB |
53
- | **Labels** | [0, 1, 2, 3, 4] |
54
- | **Number of Classes** | 5 |
55
- | **Serialization** | safetensors |
56
-
57
- ## Training Configuration
58
-
59
- | Parameter | Value |
60
- |-----------|-------|
61
- | **Batch Size** | 4 |
62
- | **Epochs** | [1, 16] |
63
- | **Training Samples** | 540 |
64
- | **Test Samples** | 100 |
65
- | **Loss Function** | CosineSimilarityLoss |
66
- | **Metric for Best Model** | embedding_loss |
67
-
68
- ### Training Progress
69
-
70
- - **Initial Loss:** 0.1474
71
- - **Final Loss:** 0.0648
72
- - **Eval Loss:** 0.0918
73
- - **Training Runtime:** 2943.9747 seconds
74
- - **Samples/Second:** 3.6690
75
-
76
- ## Evaluation Results
77
-
78
- | Metric | Score |
79
- |--------|-------|
80
- | **Accuracy** | 0.9000 |
81
- | **F1 (Weighted)** | 0.8984 |
82
- | **F1 (Macro)** | 0.8984 |
83
- | **Precision (Weighted)** | 0.9061 |
84
- | **Precision (Macro)** | 0.9061 |
85
- | **Recall (Weighted)** | 0.9000 |
86
- | **Recall (Macro)** | 0.9000 |
87
-
88
- ### Per-Class Performance
89
 
90
- ```
91
- precision recall f1-score support
92
 
93
- 0 0.86 0.95 0.90 20
94
- 1 0.83 0.75 0.79 20
95
- 2 0.83 1.00 0.91 20
96
- 3 1.00 0.80 0.89 20
97
- 4 1.00 1.00 1.00 20
98
 
99
- accuracy 0.90 100
100
- macro avg 0.91 0.90 0.90 100
101
- weighted avg 0.91 0.90 0.90 100
102
 
103
- ```
 
 
 
 
 
 
 
 
 
 
104
 
105
- ## Visualizations
106
 
107
- ### Evaluation Metrics Overview
108
- <p align="center">
109
- <img src="evaluation_metrics.png" alt="Evaluation Metrics" width="800"/>
110
- </p>
111
 
112
- ### Confusion Matrix
113
- <p align="center">
114
- <img src="confusion_matrix.png" alt="Confusion Matrix" width="600"/>
115
- </p>
116
 
117
- ### Training Loss Curve
118
- <p align="center">
119
- <img src="loss_curve.png" alt="Training Loss Curve" width="600"/>
120
- </p>
121
 
122
- ### Learning Rate Schedule
123
- <p align="center">
124
- <img src="learning_rate.png" alt="Learning Rate Schedule" width="600"/>
125
- </p>
126
 
127
- ## Usage
 
 
 
 
128
 
129
  ```python
130
  from setfit import SetFitModel
131
 
132
- # Load the model
133
- model = SetFitModel.from_pretrained("loganh274/nlp-testing-setfit")
134
-
135
- # Single prediction
136
- text = "This product exceeded my expectations!"
137
- prediction = model.predict([text])
138
- print(f"Sentiment: {prediction[0]}")
139
-
140
- # Batch prediction
141
- texts = [
142
- "Amazing quality, highly recommend!",
143
- "It's okay, nothing special.",
144
- "Terrible experience, very disappointed.",
145
- ]
146
- predictions = model.predict(texts)
147
- probabilities = model.predict_proba(texts)
148
-
149
- for text, pred, prob in zip(texts, predictions, probabilities):
150
- print(f"Text: {text}")
151
- print(f" Prediction: {pred}, Confidence: {max(prob):.2%}")
152
  ```
153
 
154
- ## Label Mapping
 
155
 
156
- | Label | Sentiment |
157
- |-------|-----------|
158
- | 0 | Negative |
159
- | 1 | Somewhat Negative |
160
- | 2 | Neutral |
161
- | 3 | Somewhat Positive |
162
- | 4 | Positive |
163
 
164
- ## Environment
 
165
 
166
- | Package | Version |
167
- |---------|---------|
168
- | Python | 3.11.14 |
169
- | SetFit | 1.1.3 |
170
- | PyTorch | 2.9.1 |
171
- | scikit-learn | 1.8.0 |
172
- | Transformers | N/A |
173
 
174
- ## Citation
 
175
 
176
- If you use this model, please cite the SetFit paper:
 
177
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
178
  ```bibtex
179
- @article{tunstall2022efficient,
180
- title={Efficient Few-Shot Learning Without Prompts},
181
- author={Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
182
- journal={arXiv preprint arXiv:2209.11055},
183
- year={2022}
 
 
 
 
184
  }
185
  ```
186
 
187
- ## License
 
 
 
 
 
 
 
 
 
 
 
 
 
188
 
189
- Apache 2.0
 
 
1
  ---
 
 
 
 
2
  tags:
3
  - setfit
4
  - sentence-transformers
5
  - text-classification
6
+ - generated_from_setfit_trainer
7
+ widget: []
 
8
  metrics:
9
  - accuracy
10
+ pipeline_tag: text-classification
11
+ library_name: setfit
12
+ inference: true
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ---
14
 
15
+ # SetFit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
+ This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
 
18
 
19
+ The model has been trained using an efficient few-shot learning technique that involves:
 
 
 
 
20
 
21
+ 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
22
+ 2. Training a classification head with features from the fine-tuned Sentence Transformer.
 
23
 
24
+ ## Model Details
25
+
26
+ ### Model Description
27
+ - **Model Type:** SetFit
28
+ <!-- - **Sentence Transformer:** [Unknown](https://huggingface.co/unknown) -->
29
+ - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
30
+ - **Maximum Sequence Length:** 512 tokens
31
+ - **Number of Classes:** 5 classes
32
+ <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
33
+ <!-- - **Language:** Unknown -->
34
+ <!-- - **License:** Unknown -->
35
 
36
+ ### Model Sources
37
 
38
+ - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
39
+ - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
40
+ - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
 
41
 
42
+ ## Uses
 
 
 
43
 
44
+ ### Direct Use for Inference
 
 
 
45
 
46
+ First install the SetFit library:
 
 
 
47
 
48
+ ```bash
49
+ pip install setfit
50
+ ```
51
+
52
+ Then you can load this model and run inference.
53
 
54
  ```python
55
  from setfit import SetFitModel
56
 
57
+ # Download from the 🤗 Hub
58
+ model = SetFitModel.from_pretrained("setfit_model_id")
59
+ # Run inference
60
+ preds = model("I loved the spiderman movie!")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
  ```
62
 
63
+ <!--
64
+ ### Downstream Use
65
 
66
+ *List how someone could finetune this model on their own dataset.*
67
+ -->
 
 
 
 
 
68
 
69
+ <!--
70
+ ### Out-of-Scope Use
71
 
72
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
73
+ -->
 
 
 
 
 
74
 
75
+ <!--
76
+ ## Bias, Risks and Limitations
77
 
78
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
79
+ -->
80
 
81
+ <!--
82
+ ### Recommendations
83
+
84
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
85
+ -->
86
+
87
+ ## Training Details
88
+
89
+ ### Framework Versions
90
+ - Python: 3.11.14
91
+ - SetFit: 1.1.3
92
+ - Sentence Transformers: 5.2.0
93
+ - Transformers: 4.57.5
94
+ - PyTorch: 2.9.1
95
+ - Datasets: 4.5.0
96
+ - Tokenizers: 0.22.2
97
+
98
+ ## Citation
99
+
100
+ ### BibTeX
101
  ```bibtex
102
+ @article{https://doi.org/10.48550/arxiv.2209.11055,
103
+ doi = {10.48550/ARXIV.2209.11055},
104
+ url = {https://arxiv.org/abs/2209.11055},
105
+ author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
106
+ keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
107
+ title = {Efficient Few-Shot Learning Without Prompts},
108
+ publisher = {arXiv},
109
+ year = {2022},
110
+ copyright = {Creative Commons Attribution 4.0 International}
111
  }
112
  ```
113
 
114
+ <!--
115
+ ## Glossary
116
+
117
+ *Clearly define terms in order to be accessible across audiences.*
118
+ -->
119
+
120
+ <!--
121
+ ## Model Card Authors
122
+
123
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
124
+ -->
125
+
126
+ <!--
127
+ ## Model Card Contact
128
 
129
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
130
+ -->
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e7ebb39c9b6f04b72a435dd467d6e4a6f8da9a408fdc023095d39b52b19a6601
3
  size 437951328
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6ce3152b5482b193511043cfd399112c1d493590764b148fb7608a344d801ed
3
  size 437951328
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e36d4eac37889fc53236f989c8cd99a56b01cb93c4aa604178f503c6f52f5388
3
  size 31647
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2c03e2c227a777526f2ab5039a4b116a2fabbd7b81dc9e31e53953535094f41
3
  size 31647