---
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: '"But PBMs operate with little to no transparency within the drug pricing
    system, and they often take advantage of their opaque position at the expense
    of patients. Their work includes establishing formularies, contracting with pharmacies,
    and negotiating rebates and discounts with drug manufacturers. But instead of
    passing these savings on to consumers, PBMs retain these costs, and the patients
    do not benefit at the pharmacy counter. But it''s actually worse than that. Just
    as a rising tide lifts all boats, PBMs'' rebate manipulation inflates health care
    prices generally and that ultimately increases the cost of patients'' medications."'
- text: '"That''s why our state''s local pharmacies are so essential. They provide
    people access to the care they need when they need it. But now, many pharmacies
    are under serious threatand our most vulnerable patients along with them. Over
    the past 14 years, the number of Oregon pharmacies has decreased more than 26%.
    Accessing medications or treatments should be simple, but unfortunately it''s
    only becoming more difficult. Why is this happening? One reason involves middlemen
    insurers called pharmacy benefit managers (PBMs)."'
- text: '"But more often, insurers and PBMs have implemented schemes called \"copay
    accumulator adjustment programs\" that prevent the value of the copay assistance
    from counting toward a patient''s deductible. Faced with unexpectedly high costs
    at the pharmacy counter, patients impacted by these policies are less likely to
    adhere to treatment which can lead to worsened health outcomes, increased hospitalizations,
    and greater costs to the health care system. Copay accumulator policies disproportionately
    impact communities of color."'
- text: '"PBMs also compile lists of drugs, called formularies, that providers of
    health benefits agree to cover; establish pharmacy networks that patients can
    access; and run their own mail-order pharmacies. Although PBMs are supposed to
    help lower costs, some of their practices may well do the opposite. PBMs often
    keep a portion of the rebates they negotiate, which can incentivize them to favor
    more expensive drugs on their formularies. (A $1 million drug, for example, would
    fetch a bigger fee than a $100 one."'
- text: '"This secrecy raises challenging questions. Do PBMs use their size and negotiating
    power to win lower net prices from drugmakers? Or do PBMs use their dominant market
    position and opaque business practices to enrich themselves at the expense of
    their customers and the rest of society? The answer to both these questions is,
    surprisingly, yes. If the contest for formulary placement works as it should,
    competition compels drugmakers to offer substantial discounts off the published
    list price. As a result, insurers and consumers benefit from a reduced net price
    for drugs. However, formulary competition can be undermined in various ways."'
metrics:
- accuracy
pipeline_tag: text-classification
library_name: setfit
inference: true
base_model: sentence-transformers/all-mpnet-base-v2
---

# SetFit with sentence-transformers/all-mpnet-base-v2

This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

## Model Details

### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 384 tokens
- **Number of Classes:** 2 classes
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

### Model Labels
| Label      | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|:-----------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Critical   | <ul><li>'"That\'s why our state\'s local pharmacies are so essential. They provide people access to the care they need when they need it. But now, many pharmacies are under serious threatand our most vulnerable patients along with them. Over the past 14 years, the number of Oregon pharmacies has decreased more than 26%. Accessing medications or treatments should be simple, but unfortunately it\'s only becoming more difficult. Why is this happening? One reason involves middlemen insurers called pharmacy benefit managers (PBMs)."'</li><li>'"Unfortunately, anti-patient policies practiced by health insurance companies and health care middlemen known as pharmacy benefit managers (PBMs) impose unnecessary access and affordability barriers for epilepsy patients ? things like fail first or step therapy requirement, prior authorization, and pocketing billions in discounts without passing savings onto patients. Many patients benefit from copay coupons and copay assistance, which often come in the form of discounts from drug manufacturers and charitable organizations to help patients afford their medicine."'</li><li>'"But PBMs operate with little to no transparency within the drug pricing system, and they often take advantage of their opaque position at the expense of patients. Their work includes establishing formularies, contracting with pharmacies, and negotiating rebates and discounts with drug manufacturers. But instead of passing these savings on to consumers, PBMs retain these costs, and the patients do not benefit at the pharmacy counter. But it\'s actually worse than that. Just as a rising tide lifts all boats, PBMs\' rebate manipulation inflates health care prices generally and that ultimately increases the cost of patients\' medications."'</li></ul>                                  |
| Supportive | <ul><li>'"Supporters of these bills claim they are about ?protecting patient choice,? but there?s not much of a choice when you can?t afford your medication to begin with. Patients don?t need laws that make it easier for Big Pharma to charge more. They need laws that encourage competition and lower prices. The average patient saves ? over $1,000 a year thanks to PBM negotiations. Take that away, and the only winner is the pharmaceutical industry. These bills don?t lower drug prices, they just shift the cost burden onto families, employers, and taxpayers. That?s not reform."'</li><li>'"This legislation, meant to punish a Pharmacy Benefit Manager, is driving up the cost of drugs for hard-working Tennesseans who were receiving their drugs at little to no cost. Not only is this in-house pharmacy losing business, but the school system is also having to include additional funding into its health insurance plan to cover additional pharmacy costs costs which were completely imposed by government action and not the rising cost of insurance. Remarkably, this means that the state government\'s actions are now being paid for by a local government."'</li><li>'"PBMs are third-party administrators of prescription medicine plans for insurance companies, businesses large and small, and government health plans. They administer the plan\'s drug formulary, process prescription claims and negotiate discounts with drug manufacturers. Basically, PBMs act as a check and balance like in our system of government on pharmaceutical companies, obtaining price discounts for the consumer in the form of rebates. Sanders\' bill would gut their ability to negotiate, under the mistaken assumption that they are the \\"bad guy,\\" and it sailed through the Senate health committee by a terrifying 18-3 vote."'</li></ul> |

## Uses

### Direct Use for Inference

First install the SetFit library:

```bash
pip install setfit
```

Then you can load this model and run inference.

```python
from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("\"PBMs also compile lists of drugs, called formularies, that providers of health benefits agree to cover; establish pharmacy networks that patients can access; and run their own mail-order pharmacies. Although PBMs are supposed to help lower costs, some of their practices may well do the opposite. PBMs often keep a portion of the rebates they negotiate, which can incentivize them to favor more expensive drugs on their formularies. (A $1 million drug, for example, would fetch a bigger fee than a $100 one.\"")
```

<!--
### Downstream Use

*List how someone could finetune this model on their own dataset.*
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set | Min | Median  | Max |
|:-------------|:----|:--------|:----|
| Word count   | 74  | 88.9474 | 100 |

| Label      | Training Sample Count |
|:-----------|:----------------------|
| Supportive | 8                     |
| Critical   | 11                    |

### Training Hyperparameters
- batch_size: (8, 8)
- num_epochs: (2, 2)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False

### Training Results
| Epoch  | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0385 | 1    | 0.201         | -               |
| 1.9231 | 50   | 0.1192        | -               |

### Framework Versions
- Python: 3.10.6
- SetFit: 1.1.1
- Sentence Transformers: 3.4.1
- Transformers: 4.50.1
- PyTorch: 2.6.0
- Datasets: 3.4.1
- Tokenizers: 0.21.1

## Citation

### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->