loganh274
/

nlp-testing-setfit

@@ -1,130 +1,189 @@
 ---
 tags:
 - setfit
 - sentence-transformers
 - text-classification
-- generated_from_setfit_trainer
-widget: []
 metrics:
 - accuracy
-pipeline_tag: text-classification
-library_name: setfit
-inference: true
 ---
-# SetFit
-This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
-The model has been trained using an efficient few-shot learning technique that involves:
-1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
-2. Training a classification head with features from the fine-tuned Sentence Transformer.
-## Model Details
-### Model Description
-- **Model Type:** SetFit
-<!-- - **Sentence Transformer:** [Unknown](https://huggingface.co/unknown) -->
-- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
-- **Maximum Sequence Length:** 512 tokens
-- **Number of Classes:** 5 classes
-<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
-### Model Sources
-- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
-- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
-- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
-## Uses
-### Direct Use for Inference
-First install the SetFit library:
-```bash
-pip install setfit
-```
-Then you can load this model and run inference.
 ```python
 from setfit import SetFitModel
-# Download from the 🤗 Hub
-model = SetFitModel.from_pretrained("setfit_model_id")
-# Run inference
-preds = model("I loved the spiderman movie!")
 ```
-<!--
-### Downstream Use
-*List how someone could finetune this model on their own dataset.*
--->
-<!--
-### Out-of-Scope Use
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
-<!--
-## Bias, Risks and Limitations
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
-<!--
-### Recommendations
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
-## Training Details
-### Framework Versions
-- Python: 3.11.14
-- SetFit: 1.1.3
-- Sentence Transformers: 5.2.0
-- Transformers: 4.57.5
-- PyTorch: 2.9.1
-- Datasets: 4.5.0
-- Tokenizers: 0.22.2
 ## Citation
-### BibTeX
 ```bibtex
-@article{https://doi.org/10.48550/arxiv.2209.11055,
-    doi = {10.48550/ARXIV.2209.11055},
-    url = {https://arxiv.org/abs/2209.11055},
-    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
-    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
-    title = {Efficient Few-Shot Learning Without Prompts},
-    publisher = {arXiv},
-    year = {2022},
-    copyright = {Creative Commons Attribution 4.0 International}
 }
 ```
-<!--
-## Glossary
-*Clearly define terms in order to be accessible across audiences.*
--->
-<!--
-## Model Card Authors
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
-<!--
-## Model Card Contact
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->

 ---
+language:
+- en
+license: apache-2.0
+library_name: setfit
 tags:
 - setfit
 - sentence-transformers
 - text-classification
+- sentiment-analysis
+- few-shot-learning
+pipeline_tag: text-classification
 metrics:
 - accuracy
+- f1
+- precision
+- recall
+model-index:
+- name: SetFit Sentiment Analysis
+  results:
+  - task:
+      type: text-classification
+      name: Sentiment Analysis
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.88
+    - name: F1 (Weighted)
+      type: f1
+      value: 0.8805050505050506
+    - name: Precision (Weighted)
+      type: precision
+      value: 0.8883333333333333
+    - name: Recall (Weighted)
+      type: recall
+      value: 0.88
 ---
+# SetFit Sentiment Analysis Model
+This is a [SetFit](https://github.com/huggingface/setfit) model fine-tuned for sentiment classification on customer feedback data.
+## Model Description
+| Property | Value |
+|----------|-------|
+| **Base Model** | [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) |
+| **Total Parameters** | 109,482,240 |
+| **Trainable Parameters** | 109,482,240 |
+| **Body Parameters** | 109,482,240 |
+| **Head Parameters** | 0 |
+| **Model Size** | 417.64 MB |
+| **Labels** | [0, 1, 2, 3, 4] |
+| **Number of Classes** | 5 |
+| **Serialization** | safetensors |
+## Training Configuration
+| Parameter | Value |
+|-----------|-------|
+| **Batch Size** | 16 |
+| **Epochs** | [1, 16] |
+| **Training Samples** | 540 |
+| **Test Samples** | 100 |
+| **Loss Function** | CosineSimilarityLoss |
+| **Metric for Best Model** | embedding_loss |
+### Training Progress
+- **Initial Loss:** 0.2366
+- **Final Loss:** 0.0893
+- **Eval Loss:** 0.0984
+- **Training Runtime:** 800.2981 seconds
+- **Samples/Second:** 13.4950
+## Evaluation Results
+| Metric | Score |
+|--------|-------|
+| **Accuracy** | 0.8800 |
+| **F1 (Weighted)** | 0.8805 |
+| **F1 (Macro)** | 0.8805 |
+| **Precision (Weighted)** | 0.8883 |
+| **Precision (Macro)** | 0.8883 |
+| **Recall (Weighted)** | 0.8800 |
+| **Recall (Macro)** | 0.8800 |
+### Per-Class Performance
+```
+              precision    recall  f1-score   support
+           0       0.90      0.90      0.90        20
+           1       0.75      0.75      0.75        20
+           2       0.79      0.95      0.86        20
+           3       1.00      0.80      0.89        20
+           4       1.00      1.00      1.00        20
+    accuracy                           0.88       100
+   macro avg       0.89      0.88      0.88       100
+weighted avg       0.89      0.88      0.88       100
+```
+## Visualizations
+### Evaluation Metrics Overview
+<p align="center">
+  <img src="evaluation_metrics.png" alt="Evaluation Metrics" width="800"/>
+</p>
+### Confusion Matrix
+<p align="center">
+  <img src="confusion_matrix.png" alt="Confusion Matrix" width="600"/>
+</p>
+### Training Loss Curve
+<p align="center">
+  <img src="loss_curve.png" alt="Training Loss Curve" width="600"/>
+</p>
+### Learning Rate Schedule
+<p align="center">
+  <img src="learning_rate.png" alt="Learning Rate Schedule" width="600"/>
+</p>
+## Usage
 ```python
 from setfit import SetFitModel
+# Load the model
+model = SetFitModel.from_pretrained("loganh274/nlp-testing-setfit")
+# Single prediction
+text = "This product exceeded my expectations!"
+prediction = model.predict([text])
+print(f"Sentiment: {prediction[0]}")
+# Batch prediction
+texts = [
+    "Amazing quality, highly recommend!",
+    "It's okay, nothing special.",
+    "Terrible experience, very disappointed.",
+]
+predictions = model.predict(texts)
+probabilities = model.predict_proba(texts)
+for text, pred, prob in zip(texts, predictions, probabilities):
+    print(f"Text: {text}")
+    print(f"  Prediction: {pred}, Confidence: {max(prob):.2%}")
 ```
+## Label Mapping
+| Label | Sentiment |
+|-------|-----------|
+| 0 | Negative |
+| 1 | Somewhat Negative |
+| 2 | Neutral |
+| 3 | Somewhat Positive |
+| 4 | Positive |
+## Environment
+| Package | Version |
+|---------|---------|
+| Python | 3.11.14 |
+| SetFit | 1.1.3 |
+| PyTorch | 2.9.1 |
+| scikit-learn | 1.8.0 |
+| Transformers | N/A |
 ## Citation
+If you use this model, please cite the SetFit paper:
 ```bibtex
+@article{tunstall2022efficient,
+  title={Efficient Few-Shot Learning Without Prompts},
+  author={Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+  journal={arXiv preprint arXiv:2209.11055},
+  year={2022}
 }
 ```
+## License
+Apache 2.0