Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,216 @@
+---
+base_model: sentence-transformers/paraphrase-mpnet-base-v2
+library_name: setfit
+metrics:
+- accuracy
+pipeline_tag: text-classification
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+widget:
+- text: I felt happy and content last night. I was with my husband and daughter and
+    we had just had dinner. We were hanging out, watching tv, eating cookies and playing
+    games. It was amazing!
+- text: 'I felt a positive emotion when I visited my friend last weekend. We had a
+    great conversation about our feelings, hopes, and aspirations. I felt present,
+    connected, and loved by someone else. '
+- text: 'I feel positive when interacting with my children.  They can be a source
+    of frustration, but they are more often a source of pride and joy.  Whenever they
+    achieve something, whether it be in sports or school, I cannot explain how bursting
+    with pride I get.  Once you have children, your whole life changes, and emotions
+    both good and bad are centered around them. '
+- text: I was able to cut my taxes in half. Also, our homeowners insurance was reduced
+    by almost 1k and we are now receiving more coverage. Additionally, I managed to
+    get our mortgage reduced from $2700 to $603.37. Quite proud of my effort(s) and
+    the results. :)
+- text: When I make a good sale at work it makes me feel so good. Also having a good
+    experience with my customers and them being happy with their purchase. It makes
+    me feel very good about my job.
+inference: true
+model-index:
+- name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: Unknown
+      type: unknown
+      split: test
+    metrics:
+    - type: accuracy
+      value: 0.6439393939393939
+      name: Accuracy
+---
+
+# SetFit with sentence-transformers/paraphrase-mpnet-base-v2
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Classes:** 5 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label                                | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| connecting with others               | <ul><li>"I was getting ready for work last night and my 14 year old son asked if I had to go to work.  I told him yes and without a response, he hugged me and held on for about thirty seconds.  I felt loved.  I was glad that he cared enough to understand that I would rather have stayed at home with him since we aren't able to spend much time together because of my job."</li><li>'I won a gift card contest about a couple weeks ago. I was so happy, because it was totally unexpected. There are over 400 employees at my workplace, but I still won. '</li><li>'The first time I held my ex girlfriends hands I felt the butterflies and smile followed. I was in complete joy beyond anything I had felt in years. I was happy excited and in complete bliss'</li></ul> |
+| overcoming challenges                | <ul><li>'I felt happy when I won a few trophies at an award ceremony at my high school. I had worked so hard throughout the year after hardships and received what I deserved. The trophies sat on the shelf and the medals were hung on the wall.'</li><li>'An example of a positive emotion occurred when I received my 3rd degree black belt. This positive emotion was one of surprise, joy, and relief because of the work I had put into it over the years, and that getting in was unexpected.'</li><li>"Happiness, hope, and empowerment is what I felt when something personal happened with my family against someone who is trying to destroy us. It's not over yet, but I hope that this person just leaves us alone."</li></ul>                                            |
+| parenthood, taking care of something | <ul><li>'I felt positive emotion when i got married to my beautiful wife and when my children were born. Life seemed complete with a positive future. Nothing could go wrong.'</li><li>'I was laying in bed watching tv one night with my 3 year old and husband when my son randomly spoke out of no where saying how he loves mama and dada and gave us both big hugs. It made my heart really happy and I was really touched with how he did it out of nowhere. It made me feel like I am doing a good job as a parent.'</li><li>'When my nephew was born I felt excitement a lot of positive emotions. I felt excitement, relief, joyful and I also felt proud of my sister. It was like I was overwhelmed by positive emotions.'</li></ul>                                         |
+| dreams and goals                     | <ul><li>' Once, I got a letter saying I got into my dream school. I felt super happy and excited because it meant my effort paid off, and I could look forward to cool things ahead. Reading the letter made me proud and thankful.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+| simple joys                          | <ul><li>'I felt a positive emotion the last time I went hiking. The emotions were happy, at peace, and energetic. Being in nature away from people, and things, and technology always calms me.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+
+## Evaluation
+
+### Metrics
+| Label   | Accuracy |
+|:--------|:---------|
+| **all** | 0.6439   |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("knharris4/harris")
+# Run inference
+preds = model("I felt happy and content last night. I was with my husband and daughter and we had just had dinner. We were hanging out, watching tv, eating cookies and playing games. It was amazing!")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median  | Max |
+|:-------------|:----|:--------|:----|
+| Word count   | 29  | 44.9444 | 85  |
+
+| Label                                | Training Sample Count |
+|:-------------------------------------|:----------------------|
+| connecting with others               | 8                     |
+| dreams and goals                     | 1                     |
+| overcoming challenges                | 4                     |
+| parenthood, taking care of something | 4                     |
+| simple joys                          | 1                     |
+
+### Training Hyperparameters
+- batch_size: (16, 16)
+- num_epochs: (1, 1)
+- max_steps: -1
+- sampling_strategy: oversampling
+- num_iterations: 10
+- body_learning_rate: (2e-05, 2e-05)
+- head_learning_rate: 2e-05
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- l2_weight: 0.01
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0435 | 1    | 0.2821        | -               |
+
+### Framework Versions
+- Python: 3.10.12
+- SetFit: 1.1.0
+- Sentence Transformers: 3.2.1
+- Transformers: 4.44.2
+- PyTorch: 2.5.0+cu121
+- Datasets: 3.0.2
+- Tokenizers: 0.19.1
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,24 @@
+{
+  "_name_or_path": "sentence-transformers/paraphrase-mpnet-base-v2",
+  "architectures": [
+    "MPNetModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2",
+  "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.2.1",
+    "transformers": "4.44.2",
+    "pytorch": "2.5.0+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

config_setfit.json

@@ -0,0 +1,10 @@
+{
+  "labels": [
+    "connecting with others",
+    "dreams and goals",
+    "overcoming challenges",
+    "parenthood, taking care of something",
+    "simple joys"
+  ],
+  "normalize_embeddings": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acf3271ec67f58afb04b196333374e83b930e64d2268d8428fdfb6a9b20f4e7a
+size 437967672

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b86da02e9f757a937f3b305172f602ccdea123a109b0c633b95c384c6af5e156
+size 32319

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,59 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30526": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "MPNetTokenizer",
+  "unk_token": "[UNK]"
+}