--- tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer widget: - text: do I still need to take my medicine? - text: send me another verification email - text: at what time do I have dinner - text: is it safe to take my insulin if my sugar is normal? - text: turn on AR guides for test sessions metrics: - accuracy pipeline_tag: text-classification library_name: setfit inference: true --- ## Ausa Hub — intent router On-device SetFit intent classifier for the **Ausa Hub** local voice assistant (see `engine/router_config.py`). Maps a patient utterance to one of **39 intents** (appointments, routines, profiles, tests/vitals, messages, meal_time, symptoms, summary, settings, and the `system.*` catch-alls). Runs as ONNX on-device and also seeds the cloud LLM router's priors. - **Intents:** 39 - **Training examples:** ~1699 - **Last updated:** 2026-06-25 - **Recent change:** conversation-meta / recall utterances ("what did I just say", "what did I tell you my dog's name was", "remind me what I said") and general-knowledge questions now classify as `system.oos` so they reach the conversational assistant instead of a records intent. # SetFit This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 39 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | messages.send | | | appointments.read | | | profiles.care_team | | | system.oos | | | routines.update | | | summary.read | | | tests.take_test | | | tests.test_history | | | profiles.family.update | | | symptoms.report | | | meal_time.update | | | meal_time.read | | | profiles.personal_info.picture | | | settings.calls | | | routines.read | | | profiles.read | | | profiles.personal_info.read | | | profiles.allergies.update | | | appointments.create | | | profiles.update | | | routines.create | | | settings.wifi | | | profiles.family.read | | | profiles.family.create | | | messages.read | | | profiles.personal_info.verify_email | | | profiles.family.delete | | | appointments.delete | | | routines.delete | | | settings.display | | | settings.notifications | | | system.cancel | | | profiles.allergies.read | | | profiles.allergies.delete | | | settings.devices | | | profiles.personal_info.update | | | appointments.update | | | profiles.personal_info.verify_phone | | | profiles.allergies.create | | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("setfit_model_id") # Run inference preds = model("at what time do I have dinner") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:-------|:----| | Word count | 1 | 6.2825 | 21 | | Label | Training Sample Count | |:------------------------------------|:----------------------| | appointments.create | 27 | | appointments.delete | 37 | | appointments.read | 63 | | appointments.update | 24 | | meal_time.read | 57 | | meal_time.update | 26 | | messages.read | 39 | | messages.send | 109 | | profiles.allergies.create | 22 | | profiles.allergies.delete | 26 | | profiles.allergies.read | 22 | | profiles.allergies.update | 22 | | profiles.care_team | 40 | | profiles.family.create | 29 | | profiles.family.delete | 21 | | profiles.family.read | 35 | | profiles.family.update | 21 | | profiles.personal_info.picture | 21 | | profiles.personal_info.read | 36 | | profiles.personal_info.update | 28 | | profiles.personal_info.verify_email | 22 | | profiles.personal_info.verify_phone | 22 | | profiles.read | 28 | | profiles.update | 21 | | routines.create | 42 | | routines.delete | 24 | | routines.read | 51 | | routines.update | 24 | | settings.calls | 21 | | settings.devices | 22 | | settings.display | 28 | | settings.notifications | 22 | | settings.wifi | 22 | | summary.read | 32 | | symptoms.report | 44 | | system.cancel | 27 | | system.oos | 180 | | tests.take_test | 31 | | tests.test_history | 76 | ### Training Hyperparameters - batch_size: (16, 16) - num_epochs: (3, 3) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 20 - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 0.01 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - l2_weight: 0.01 - seed: 42 - evaluation_strategy: epoch - eval_max_steps: -1 - load_best_model_at_end: True ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:------:|:-----:|:-------------:|:---------------:| | 0.0003 | 1 | 0.0006 | - | | 0.0139 | 50 | 0.0042 | - | | 0.0277 | 100 | 0.0011 | - | | 0.0416 | 150 | 0.0009 | - | | 0.0554 | 200 | 0.0004 | - | | 0.0693 | 250 | 0.0005 | - | | 0.0831 | 300 | 0.0004 | - | | 0.0970 | 350 | 0.0005 | - | | 0.1108 | 400 | 0.0004 | - | | 0.1247 | 450 | 0.0004 | - | | 0.1385 | 500 | 0.0005 | - | | 0.1524 | 550 | 0.0004 | - | | 0.1662 | 600 | 0.0004 | - | | 0.1801 | 650 | 0.0004 | - | | 0.1939 | 700 | 0.0004 | - | | 0.2078 | 750 | 0.0004 | - | | 0.2216 | 800 | 0.0004 | - | | 0.2355 | 850 | 0.0005 | - | | 0.2493 | 900 | 0.0005 | - | | 0.2632 | 950 | 0.0005 | - | | 0.2770 | 1000 | 0.0005 | - | | 0.2909 | 1050 | 0.0005 | - | | 0.3047 | 1100 | 0.0005 | - | | 0.3186 | 1150 | 0.0005 | - | | 0.3324 | 1200 | 0.0006 | - | | 0.3463 | 1250 | 0.0005 | - | | 0.3601 | 1300 | 0.0005 | - | | 0.3740 | 1350 | 0.0005 | - | | 0.3878 | 1400 | 0.0032 | - | | 0.4017 | 1450 | 0.0006 | - | | 0.4155 | 1500 | 0.0006 | - | | 0.4294 | 1550 | 0.0005 | - | | 0.4432 | 1600 | 0.0005 | - | | 0.4571 | 1650 | 0.0005 | - | | 0.4709 | 1700 | 0.0007 | - | | 0.4848 | 1750 | 0.0005 | - | | 0.4986 | 1800 | 0.0005 | - | | 0.5125 | 1850 | 0.0005 | - | | 0.5263 | 1900 | 0.0006 | - | | 0.5402 | 1950 | 0.0005 | - | | 0.5540 | 2000 | 0.0005 | - | | 0.5679 | 2050 | 0.0005 | - | | 0.5817 | 2100 | 0.0005 | - | | 0.5956 | 2150 | 0.0006 | - | | 0.6094 | 2200 | 0.0005 | - | | 0.6233 | 2250 | 0.0004 | - | | 0.6371 | 2300 | 0.0005 | - | | 0.6510 | 2350 | 0.0005 | - | | 0.6648 | 2400 | 0.0005 | - | | 0.6787 | 2450 | 0.0005 | - | | 0.6925 | 2500 | 0.0005 | - | | 0.7064 | 2550 | 0.0005 | - | | 0.7202 | 2600 | 0.0004 | - | | 0.7341 | 2650 | 0.0013 | - | | 0.7479 | 2700 | 0.0005 | - | | 0.7618 | 2750 | 0.0005 | - | | 0.7756 | 2800 | 0.0005 | - | | 0.7895 | 2850 | 0.0004 | - | | 0.8033 | 2900 | 0.0005 | - | | 0.8172 | 2950 | 0.0005 | - | | 0.8310 | 3000 | 0.0004 | - | | 0.8449 | 3050 | 0.0005 | - | | 0.8587 | 3100 | 0.0007 | - | | 0.8726 | 3150 | 0.0004 | - | | 0.8864 | 3200 | 0.0005 | - | | 0.9003 | 3250 | 0.0005 | - | | 0.9141 | 3300 | 0.0005 | - | | 0.9280 | 3350 | 0.0004 | - | | 0.9418 | 3400 | 0.0006 | - | | 0.9557 | 3450 | 0.0004 | - | | 0.9695 | 3500 | 0.0005 | - | | 0.9834 | 3550 | 0.0004 | - | | 0.9972 | 3600 | 0.0008 | - | | 1.0 | 3610 | - | 0.0001 | | 1.0111 | 3650 | 0.0004 | - | | 1.0249 | 3700 | 0.0006 | - | | 1.0388 | 3750 | 0.0005 | - | | 1.0526 | 3800 | 0.0004 | - | | 1.0665 | 3850 | 0.0007 | - | | 1.0803 | 3900 | 0.0005 | - | | 1.0942 | 3950 | 0.0004 | - | | 1.1080 | 4000 | 0.0005 | - | | 1.1219 | 4050 | 0.0004 | - | | 1.1357 | 4100 | 0.0005 | - | | 1.1496 | 4150 | 0.0004 | - | | 1.1634 | 4200 | 0.0008 | - | | 1.1773 | 4250 | 0.0004 | - | | 1.1911 | 4300 | 0.0004 | - | | 1.2050 | 4350 | 0.0004 | - | | 1.2188 | 4400 | 0.0004 | - | | 1.2327 | 4450 | 0.0004 | - | | 1.2465 | 4500 | 0.0005 | - | | 1.2604 | 4550 | 0.0004 | - | | 1.2742 | 4600 | 0.0004 | - | | 1.2881 | 4650 | 0.0005 | - | | 1.3019 | 4700 | 0.0004 | - | | 1.3158 | 4750 | 0.0005 | - | | 1.3296 | 4800 | 0.0004 | - | | 1.3435 | 4850 | 0.0004 | - | | 1.3573 | 4900 | 0.0004 | - | | 1.3712 | 4950 | 0.0004 | - | | 1.3850 | 5000 | 0.0005 | - | | 1.3989 | 5050 | 0.0004 | - | | 1.4127 | 5100 | 0.0004 | - | | 1.4266 | 5150 | 0.0004 | - | | 1.4404 | 5200 | 0.0007 | - | | 1.4543 | 5250 | 0.0004 | - | | 1.4681 | 5300 | 0.0004 | - | | 1.4820 | 5350 | 0.0004 | - | | 1.4958 | 5400 | 0.0005 | - | | 1.5097 | 5450 | 0.0004 | - | | 1.5235 | 5500 | 0.0005 | - | | 1.5374 | 5550 | 0.0004 | - | | 1.5512 | 5600 | 0.0005 | - | | 1.5651 | 5650 | 0.0004 | - | | 1.5789 | 5700 | 0.0004 | - | | 1.5928 | 5750 | 0.0004 | - | | 1.6066 | 5800 | 0.0004 | - | | 1.6205 | 5850 | 0.0004 | - | | 1.6343 | 5900 | 0.0004 | - | | 1.6482 | 5950 | 0.0004 | - | | 1.6620 | 6000 | 0.0005 | - | | 1.6759 | 6050 | 0.0005 | - | | 1.6898 | 6100 | 0.0004 | - | | 1.7036 | 6150 | 0.0012 | - | | 1.7175 | 6200 | 0.0007 | - | | 1.7313 | 6250 | 0.0006 | - | | 1.7452 | 6300 | 0.0015 | - | | 1.7590 | 6350 | 0.0005 | - | | 1.7729 | 6400 | 0.0004 | - | | 1.7867 | 6450 | 0.0005 | - | | 1.8006 | 6500 | 0.0017 | - | | 1.8144 | 6550 | 0.0004 | - | | 1.8283 | 6600 | 0.0004 | - | | 1.8421 | 6650 | 0.0004 | - | | 1.8560 | 6700 | 0.0004 | - | | 1.8698 | 6750 | 0.0004 | - | | 1.8837 | 6800 | 0.0004 | - | | 1.8975 | 6850 | 0.0004 | - | | 1.9114 | 6900 | 0.0004 | - | | 1.9252 | 6950 | 0.0004 | - | | 1.9391 | 7000 | 0.0004 | - | | 1.9529 | 7050 | 0.0004 | - | | 1.9668 | 7100 | 0.0004 | - | | 1.9806 | 7150 | 0.0004 | - | | 1.9945 | 7200 | 0.0004 | - | | 2.0 | 7220 | - | 0.0001 | | 2.0083 | 7250 | 0.0005 | - | | 2.0222 | 7300 | 0.0004 | - | | 2.0360 | 7350 | 0.0006 | - | | 2.0499 | 7400 | 0.0004 | - | | 2.0637 | 7450 | 0.0004 | - | | 2.0776 | 7500 | 0.0003 | - | | 2.0914 | 7550 | 0.0004 | - | | 2.1053 | 7600 | 0.0005 | - | | 2.1191 | 7650 | 0.0004 | - | | 2.1330 | 7700 | 0.0004 | - | | 2.1468 | 7750 | 0.0004 | - | | 2.1607 | 7800 | 0.0004 | - | | 2.1745 | 7850 | 0.0004 | - | | 2.1884 | 7900 | 0.0004 | - | | 2.2022 | 7950 | 0.0004 | - | | 2.2161 | 8000 | 0.0004 | - | | 2.2299 | 8050 | 0.0005 | - | | 2.2438 | 8100 | 0.0004 | - | | 2.2576 | 8150 | 0.0004 | - | | 2.2715 | 8200 | 0.0004 | - | | 2.2853 | 8250 | 0.0004 | - | | 2.2992 | 8300 | 0.0004 | - | | 2.3130 | 8350 | 0.0004 | - | | 2.3269 | 8400 | 0.0004 | - | | 2.3407 | 8450 | 0.0004 | - | | 2.3546 | 8500 | 0.0004 | - | | 2.3684 | 8550 | 0.0004 | - | | 2.3823 | 8600 | 0.0007 | - | | 2.3961 | 8650 | 0.0004 | - | | 2.4100 | 8700 | 0.0004 | - | | 2.4238 | 8750 | 0.0004 | - | | 2.4377 | 8800 | 0.0004 | - | | 2.4515 | 8850 | 0.0003 | - | | 2.4654 | 8900 | 0.0003 | - | | 2.4792 | 8950 | 0.0004 | - | | 2.4931 | 9000 | 0.0004 | - | | 2.5069 | 9050 | 0.0004 | - | | 2.5208 | 9100 | 0.0006 | - | | 2.5346 | 9150 | 0.0004 | - | | 2.5485 | 9200 | 0.0004 | - | | 2.5623 | 9250 | 0.0004 | - | | 2.5762 | 9300 | 0.0004 | - | | 2.5900 | 9350 | 0.0003 | - | | 2.6039 | 9400 | 0.0004 | - | | 2.6177 | 9450 | 0.0003 | - | | 2.6316 | 9500 | 0.0004 | - | | 2.6454 | 9550 | 0.0003 | - | | 2.6593 | 9600 | 0.0004 | - | | 2.6731 | 9650 | 0.0004 | - | | 2.6870 | 9700 | 0.0004 | - | | 2.7008 | 9750 | 0.0003 | - | | 2.7147 | 9800 | 0.0003 | - | | 2.7285 | 9850 | 0.0003 | - | | 2.7424 | 9900 | 0.0004 | - | | 2.7562 | 9950 | 0.0003 | - | | 2.7701 | 10000 | 0.0003 | - | | 2.7839 | 10050 | 0.0005 | - | | 2.7978 | 10100 | 0.0004 | - | | 2.8116 | 10150 | 0.0003 | - | | 2.8255 | 10200 | 0.0003 | - | | 2.8393 | 10250 | 0.0003 | - | | 2.8532 | 10300 | 0.0003 | - | | 2.8670 | 10350 | 0.0004 | - | | 2.8809 | 10400 | 0.0003 | - | | 2.8947 | 10450 | 0.0004 | - | | 2.9086 | 10500 | 0.0003 | - | | 2.9224 | 10550 | 0.0004 | - | | 2.9363 | 10600 | 0.0004 | - | | 2.9501 | 10650 | 0.0004 | - | | 2.9640 | 10700 | 0.0003 | - | | 2.9778 | 10750 | 0.0003 | - | | 2.9917 | 10800 | 0.0003 | - | | 3.0 | 10830 | - | 0.0000 | ### Framework Versions - Python: 3.12.10 - SetFit: 1.1.3 - Sentence Transformers: 5.4.1 - Transformers: 4.57.6 - PyTorch: 2.11.0 - Datasets: 4.8.4 - Tokenizers: 0.22.2 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```