aadiausa commited on
Commit
59654d0
·
verified ·
1 Parent(s): 499900f

Retrain: recall/meta + general questions -> system.oos (2026-06-25); refresh model card

Browse files
Files changed (3) hide show
  1. README.md +286 -263
  2. model.safetensors +1 -1
  3. model_head.pkl +1 -1
README.md CHANGED
@@ -5,11 +5,11 @@ tags:
5
  - text-classification
6
  - generated_from_setfit_trainer
7
  widget:
8
- - text: when am I eating breakfast
9
- - text: scratch my Monday morning appointment
10
- - text: validate my phone number with OTP
11
- - text: Unpair my old smartwatch from the system, I don't use it anymore.
12
- - text: I won't be there for my appointment next week
13
  metrics:
14
  - accuracy
15
  pipeline_tag: text-classification
@@ -17,6 +17,22 @@ library_name: setfit
17
  inference: true
18
  ---
19
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  # SetFit
21
 
22
  This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
@@ -45,47 +61,47 @@ The model has been trained using an efficient few-shot learning technique that i
45
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
46
 
47
  ### Model Labels
48
- | Label | Examples |
49
- |:------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
50
- | appointments.delete | <ul><li>'take my Thursday appointment off the calendar'</li><li>"Scrap the follow-up appointment with the dermatologist, I won't be needing it."</li><li>'cancel the telehealth call I scheduled'</li></ul> |
51
- | meal_time.read | <ul><li>'what time do I have my meals'</li><li>'when am I eating dinner'</li><li>'tell me my meal schedule'</li></ul> |
52
- | system.oos | <ul><li>'is it going to rain'</li><li>'that one'</li><li>'tell me what you can do'</li></ul> |
53
- | settings.notifications | <ul><li>'I want to make sure I get pinged whenever my care team sends a message, turn those on.'</li><li>'what notification types do I have turned on'</li><li>'stop sending me email alerts'</li></ul> |
54
- | settings.display | <ul><li>'brighten the display'</li><li>'set brightness to 50 percent'</li><li>'increase the font size on screen'</li></ul> |
55
- | profiles.family.delete | <ul><li>'kick my uncle off my account'</li><li>'remove my aunt from family sharing'</li><li>"Revoke my ex-wife's access to all my medical data immediately."</li></ul> |
56
- | routines.delete | <ul><li>'I want to remove my stretching routine'</li><li>'cancel my blood pressure testing routine'</li><li>'forget it, remove the gym routine'</li></ul> |
57
- | profiles.personal_info.verify_email | <ul><li>'validate my email with a code'</li><li>'can you confirm my email is valid'</li><li>'Send the confirmation link to my Gmail so I can validate my account.'</li></ul> |
58
- | appointments.create | <ul><li>'I need to squeeze in a quick telehealth visit for a prescription refill before Friday.'</li><li>'book a new appointment with my doctor'</li><li>'schedule an appointment for today at 4pm I have been feeling dizzy'</li></ul> |
59
- | tests.take_test | <ul><li>'leave it, I want to measure my blood pressure'</li><li>'check my oxygen level now'</li><li>'start the glucose monitoring'</li></ul> |
60
- | symptoms.report | <ul><li>'I feel a sharp pain when I breathe'</li><li>'My stomach hurts'</li><li>'My back has been hurting really bad'</li></ul> |
61
- | appointments.read | <ul><li>'any open times today'</li><li>'list all my scheduled visits'</li><li>'is anything scheduled for this Friday'</li></ul> |
62
- | system.cancel | <ul><li>'I want to stop'</li><li>'disregard'</li><li>'forget it'</li></ul> |
63
- | profiles.family.create | <ul><li>'invite someone to my account'</li><li>'forget this, add my wife to my account'</li><li>'link my stepfather to my health data'</li></ul> |
64
- | tests.test_history | <ul><li>'nevermind, show my test history'</li><li>'Can you find the results of that lung sound test I did back in early October?'</li><li>'is my body temperature too high'</li></ul> |
65
- | meal_time.update | <ul><li>'change my breakfast time'</li><li>'set my dinner meal time to 8pm'</li><li>'change my dinner to 8pm'</li></ul> |
66
- | profiles.care_team | <ul><li>'can you tell me about my healthcare team'</li><li>'nevermind, who is my doctor'</li><li>'who is on my care team'</li></ul> |
67
- | routines.update | <ul><li>'switch my daily walk from 7am to 8am'</li><li>'move my supplement routine from morning to night'</li><li>'reschedule my blood pressure routine to 9am starting next Monday'</li></ul> |
68
- | messages.send | <ul><li>'send a message to my care team about my symptoms'</li><li>'take a picture and send it to my doctor'</li><li>"notify my doctor I'm experiencing chest tightness"</li></ul> |
69
- | routines.read | <ul><li>'tell me about my exercise routines'</li><li>'show my routine schedule for this weekend'</li><li>'when is my next blood pressure test'</li></ul> |
70
- | settings.devices | <ul><li>'remove the old thermometer from my devices'</li><li>'what devices are currently connected to my account'</li><li>'set up my new heart rate monitor'</li></ul> |
71
- | profiles.read | <ul><li>'show me my health profile'</li><li>'display my profile overview'</li><li>'forget it, pull up my health profile'</li></ul> |
72
- | summary.read | <ul><li>"how's my day"</li><li>'what should I expect today'</li><li>"what's my outlook for today"</li></ul> |
73
- | profiles.family.read | <ul><li>'who has access to my account'</li><li>'read my family information'</li><li>'does my daughter have access to my routines'</li></ul> |
74
- | profiles.allergies.create | <ul><li>'I found out I am allergic to something new'</li><li>'I recently discovered I am allergic to strawberries'</li><li>'add a bee venom allergy high severity with anaphylaxis risk'</li></ul> |
75
- | settings.calls | <ul><li>'what are my current call preferences'</li><li>'enable background blur and closed captions'</li><li>'hide video during tests'</li></ul> |
76
- | profiles.personal_info.update | <ul><li>'I moved to a new city update my zip code'</li><li>'I got a new cell phone, please update my contact number to 555-0192.'</li><li>'I have a new phone number please update it'</li></ul> |
77
- | appointments.update | <ul><li>'I need to change the time of my visit'</li><li>'move my morning appointment to the afternoon'</li><li>'change my cardiology appointment from the 3rd to the 7th'</li></ul> |
78
- | profiles.personal_info.verify_phone | <ul><li>'start the phone verification process'</li><li>'check if my phone number has been verified'</li><li>'how do I verify my phone on this account'</li></ul> |
79
- | profiles.family.update | <ul><li>'allow my uncle to see my blood pressure readings'</li><li>"extend my son's data sharing access for another 30 days"</li><li>"change my mother's access from view only to full access"</li></ul> |
80
- | profiles.personal_info.picture | <ul><li>'update my avatar image'</li><li>'replace my profile image'</li><li>'update my profile photo'</li></ul> |
81
- | profiles.allergies.delete | <ul><li>'delete an old allergy that is not relevant'</li><li>'remove the ibuprofen allergy please'</li><li>'take the bee venom allergy off my chart'</li></ul> |
82
- | routines.create | <ul><li>'remind me to take my tablets after lunch'</li><li>'I want to add a routine for my oxygen level checks every other day at noon'</li><li>'remind me to do stretching exercises daily at 6:30am for 20 minutes'</li></ul> |
83
- | profiles.allergies.read | <ul><li>'what am I allergic to'</li><li>'do I have any allergies recorded'</li><li>'list all of my allergies'</li></ul> |
84
- | profiles.personal_info.read | <ul><li>'display my personal details'</li><li>'show me my profile information'</li><li>'what conditions do I have on my profile'</li></ul> |
85
- | messages.read | <ul><li>'what did Dr. Chen say'</li><li>'show me my recent messages'</li><li>'show my latest messages'</li></ul> |
86
- | settings.wifi | <ul><li>'show available wifi networks'</li><li>'add a new wifi network'</li><li>"I need to connect this thing to my home network, it's called 'Netgear5G'."</li></ul> |
87
- | profiles.update | <ul><li>'delete my peanut allergy'</li><li>'add a new medical condition to my profile'</li><li>'edit my medical history'</li></ul> |
88
- | profiles.allergies.update | <ul><li>'change my tree nut allergy reaction from mild to moderate'</li><li>'my sulfa drug allergy is now high severity please update it'</li><li>'update the severity of my dust allergy from low to moderate'</li></ul> |
89
 
90
  ## Uses
91
 
@@ -105,7 +121,7 @@ from setfit import SetFitModel
105
  # Download from the 🤗 Hub
106
  model = SetFitModel.from_pretrained("setfit_model_id")
107
  # Run inference
108
- preds = model("when am I eating breakfast")
109
  ```
110
 
111
  <!--
@@ -137,7 +153,7 @@ preds = model("when am I eating breakfast")
137
  ### Training Set Metrics
138
  | Training set | Min | Median | Max |
139
  |:-------------|:----|:-------|:----|
140
- | Word count | 1 | 6.2773 | 21 |
141
 
142
  | Label | Training Sample Count |
143
  |:------------------------------------|:----------------------|
@@ -150,7 +166,7 @@ preds = model("when am I eating breakfast")
150
  | messages.read | 39 |
151
  | messages.send | 109 |
152
  | profiles.allergies.create | 22 |
153
- | profiles.allergies.delete | 25 |
154
  | profiles.allergies.read | 22 |
155
  | profiles.allergies.update | 22 |
156
  | profiles.care_team | 40 |
@@ -177,9 +193,9 @@ preds = model("when am I eating breakfast")
177
  | summary.read | 32 |
178
  | symptoms.report | 44 |
179
  | system.cancel | 27 |
180
- | system.oos | 146 |
181
  | tests.take_test | 31 |
182
- | tests.test_history | 66 |
183
 
184
  ### Training Hyperparameters
185
  - batch_size: (16, 16)
@@ -205,218 +221,225 @@ preds = model("when am I eating breakfast")
205
  | Epoch | Step | Training Loss | Validation Loss |
206
  |:------:|:-----:|:-------------:|:---------------:|
207
  | 0.0003 | 1 | 0.0006 | - |
208
- | 0.0143 | 50 | 0.0072 | - |
209
- | 0.0286 | 100 | 0.0165 | - |
210
- | 0.0429 | 150 | 0.0048 | - |
211
- | 0.0572 | 200 | 0.0087 | - |
212
- | 0.0715 | 250 | 0.0054 | - |
213
- | 0.0858 | 300 | 0.0017 | - |
214
- | 0.1001 | 350 | 0.0059 | - |
215
- | 0.1144 | 400 | 0.0025 | - |
216
- | 0.1286 | 450 | 0.0029 | - |
217
- | 0.1429 | 500 | 0.0021 | - |
218
- | 0.1572 | 550 | 0.0057 | - |
219
- | 0.1715 | 600 | 0.0032 | - |
220
- | 0.1858 | 650 | 0.0017 | - |
221
- | 0.2001 | 700 | 0.0054 | - |
222
- | 0.2144 | 750 | 0.0029 | - |
223
- | 0.2287 | 800 | 0.0004 | - |
224
- | 0.2430 | 850 | 0.0029 | - |
225
- | 0.2573 | 900 | 0.0041 | - |
226
- | 0.2716 | 950 | 0.0017 | - |
227
- | 0.2859 | 1000 | 0.003 | - |
228
- | 0.3002 | 1050 | 0.0017 | - |
229
- | 0.3145 | 1100 | 0.0017 | - |
230
- | 0.3288 | 1150 | 0.0033 | - |
231
- | 0.3431 | 1200 | 0.003 | - |
232
- | 0.3573 | 1250 | 0.0017 | - |
233
- | 0.3716 | 1300 | 0.0041 | - |
234
- | 0.3859 | 1350 | 0.0018 | - |
235
- | 0.4002 | 1400 | 0.0053 | - |
236
- | 0.4145 | 1450 | 0.0029 | - |
237
- | 0.4288 | 1500 | 0.0041 | - |
238
- | 0.4431 | 1550 | 0.0005 | - |
239
- | 0.4574 | 1600 | 0.0018 | - |
240
- | 0.4717 | 1650 | 0.0017 | - |
241
- | 0.4860 | 1700 | 0.0053 | - |
242
- | 0.5003 | 1750 | 0.0021 | - |
243
- | 0.5146 | 1800 | 0.0005 | - |
244
- | 0.5289 | 1850 | 0.0041 | - |
245
- | 0.5432 | 1900 | 0.0024 | - |
246
- | 0.5575 | 1950 | 0.0053 | - |
247
- | 0.5718 | 2000 | 0.0005 | - |
248
- | 0.5860 | 2050 | 0.003 | - |
249
- | 0.6003 | 2100 | 0.0029 | - |
250
- | 0.6146 | 2150 | 0.0029 | - |
251
- | 0.6289 | 2200 | 0.0072 | - |
252
- | 0.6432 | 2250 | 0.0035 | - |
253
- | 0.6575 | 2300 | 0.0043 | - |
254
- | 0.6718 | 2350 | 0.0055 | - |
255
- | 0.6861 | 2400 | 0.0041 | - |
256
- | 0.7004 | 2450 | 0.0053 | - |
257
- | 0.7147 | 2500 | 0.0042 | - |
258
- | 0.7290 | 2550 | 0.0055 | - |
259
- | 0.7433 | 2600 | 0.0034 | - |
260
- | 0.7576 | 2650 | 0.003 | - |
261
- | 0.7719 | 2700 | 0.0041 | - |
262
- | 0.7862 | 2750 | 0.0006 | - |
263
- | 0.8005 | 2800 | 0.0042 | - |
264
- | 0.8148 | 2850 | 0.0041 | - |
265
- | 0.8290 | 2900 | 0.0055 | - |
266
- | 0.8433 | 2950 | 0.0056 | - |
267
- | 0.8576 | 3000 | 0.0019 | - |
268
- | 0.8719 | 3050 | 0.0005 | - |
269
- | 0.8862 | 3100 | 0.0032 | - |
270
- | 0.9005 | 3150 | 0.0032 | - |
271
- | 0.9148 | 3200 | 0.0041 | - |
272
- | 0.9291 | 3250 | 0.0063 | - |
273
- | 0.9434 | 3300 | 0.0016 | - |
274
- | 0.9577 | 3350 | 0.0098 | - |
275
- | 0.9720 | 3400 | 0.0057 | - |
276
- | 0.9863 | 3450 | 0.0054 | - |
277
- | 1.0 | 3498 | - | 0.0113 |
278
- | 1.0006 | 3500 | 0.0039 | - |
279
- | 1.0149 | 3550 | 0.0051 | - |
280
- | 1.0292 | 3600 | 0.0025 | - |
281
- | 1.0435 | 3650 | 0.0023 | - |
282
- | 1.0577 | 3700 | 0.0026 | - |
283
- | 1.0720 | 3750 | 0.0016 | - |
284
- | 1.0863 | 3800 | 0.0015 | - |
285
- | 1.1006 | 3850 | 0.0012 | - |
286
- | 1.1149 | 3900 | 0.0017 | - |
287
- | 1.1292 | 3950 | 0.0008 | - |
288
- | 1.1435 | 4000 | 0.0012 | - |
289
- | 1.1578 | 4050 | 0.0006 | - |
290
- | 1.1721 | 4100 | 0.0017 | - |
291
- | 1.1864 | 4150 | 0.0006 | - |
292
- | 1.2007 | 4200 | 0.0006 | - |
293
- | 1.2150 | 4250 | 0.0016 | - |
294
- | 1.2293 | 4300 | 0.0014 | - |
295
- | 1.2436 | 4350 | 0.0007 | - |
296
- | 1.2579 | 4400 | 0.0005 | - |
297
- | 1.2722 | 4450 | 0.0006 | - |
298
- | 1.2864 | 4500 | 0.0006 | - |
299
- | 1.3007 | 4550 | 0.0006 | - |
300
- | 1.3150 | 4600 | 0.0015 | - |
301
- | 1.3293 | 4650 | 0.0014 | - |
302
- | 1.3436 | 4700 | 0.0007 | - |
303
- | 1.3579 | 4750 | 0.0007 | - |
304
- | 1.3722 | 4800 | 0.0012 | - |
305
- | 1.3865 | 4850 | 0.0009 | - |
306
- | 1.4008 | 4900 | 0.0006 | - |
307
- | 1.4151 | 4950 | 0.0008 | - |
308
- | 1.4294 | 5000 | 0.0008 | - |
309
- | 1.4437 | 5050 | 0.0006 | - |
310
- | 1.4580 | 5100 | 0.0006 | - |
311
- | 1.4723 | 5150 | 0.0011 | - |
312
- | 1.4866 | 5200 | 0.0009 | - |
313
- | 1.5009 | 5250 | 0.0007 | - |
314
- | 1.5152 | 5300 | 0.0006 | - |
315
- | 1.5294 | 5350 | 0.0006 | - |
316
- | 1.5437 | 5400 | 0.0008 | - |
317
- | 1.5580 | 5450 | 0.0008 | - |
318
- | 1.5723 | 5500 | 0.0008 | - |
319
- | 1.5866 | 5550 | 0.0006 | - |
320
- | 1.6009 | 5600 | 0.0007 | - |
321
- | 1.6152 | 5650 | 0.0006 | - |
322
- | 1.6295 | 5700 | 0.0006 | - |
323
- | 1.6438 | 5750 | 0.0005 | - |
324
- | 1.6581 | 5800 | 0.0006 | - |
325
- | 1.6724 | 5850 | 0.0006 | - |
326
- | 1.6867 | 5900 | 0.0006 | - |
327
- | 1.7010 | 5950 | 0.0006 | - |
328
- | 1.7153 | 6000 | 0.0007 | - |
329
- | 1.7296 | 6050 | 0.0005 | - |
330
- | 1.7439 | 6100 | 0.0005 | - |
331
- | 1.7581 | 6150 | 0.0009 | - |
332
- | 1.7724 | 6200 | 0.0005 | - |
333
- | 1.7867 | 6250 | 0.0008 | - |
334
- | 1.8010 | 6300 | 0.0005 | - |
335
- | 1.8153 | 6350 | 0.0005 | - |
336
- | 1.8296 | 6400 | 0.0006 | - |
337
- | 1.8439 | 6450 | 0.0005 | - |
338
- | 1.8582 | 6500 | 0.0005 | - |
339
- | 1.8725 | 6550 | 0.0006 | - |
340
- | 1.8868 | 6600 | 0.0008 | - |
341
- | 1.9011 | 6650 | 0.0005 | - |
342
- | 1.9154 | 6700 | 0.0005 | - |
343
- | 1.9297 | 6750 | 0.0007 | - |
344
- | 1.9440 | 6800 | 0.0006 | - |
345
- | 1.9583 | 6850 | 0.0006 | - |
346
- | 1.9726 | 6900 | 0.0006 | - |
347
- | 1.9868 | 6950 | 0.0005 | - |
348
- | 2.0 | 6996 | - | 0.0096 |
349
- | 2.0011 | 7000 | 0.0005 | - |
350
- | 2.0154 | 7050 | 0.0004 | - |
351
- | 2.0297 | 7100 | 0.0005 | - |
352
- | 2.0440 | 7150 | 0.0005 | - |
353
- | 2.0583 | 7200 | 0.0005 | - |
354
- | 2.0726 | 7250 | 0.0005 | - |
355
- | 2.0869 | 7300 | 0.0005 | - |
356
- | 2.1012 | 7350 | 0.0006 | - |
357
- | 2.1155 | 7400 | 0.0004 | - |
358
- | 2.1298 | 7450 | 0.0006 | - |
359
- | 2.1441 | 7500 | 0.0005 | - |
360
- | 2.1584 | 7550 | 0.0005 | - |
361
- | 2.1727 | 7600 | 0.0006 | - |
362
- | 2.1870 | 7650 | 0.0006 | - |
363
- | 2.2013 | 7700 | 0.0005 | - |
364
- | 2.2156 | 7750 | 0.0006 | - |
365
- | 2.2298 | 7800 | 0.0006 | - |
366
- | 2.2441 | 7850 | 0.0004 | - |
367
- | 2.2584 | 7900 | 0.0004 | - |
368
- | 2.2727 | 7950 | 0.0004 | - |
369
- | 2.2870 | 8000 | 0.0005 | - |
370
- | 2.3013 | 8050 | 0.0005 | - |
371
- | 2.3156 | 8100 | 0.0004 | - |
372
- | 2.3299 | 8150 | 0.0005 | - |
373
- | 2.3442 | 8200 | 0.0005 | - |
374
- | 2.3585 | 8250 | 0.0007 | - |
375
- | 2.3728 | 8300 | 0.0005 | - |
376
- | 2.3871 | 8350 | 0.0004 | - |
377
- | 2.4014 | 8400 | 0.0007 | - |
378
- | 2.4157 | 8450 | 0.0006 | - |
379
- | 2.4300 | 8500 | 0.0005 | - |
380
- | 2.4443 | 8550 | 0.0006 | - |
381
- | 2.4585 | 8600 | 0.0005 | - |
382
- | 2.4728 | 8650 | 0.0004 | - |
383
- | 2.4871 | 8700 | 0.0005 | - |
384
- | 2.5014 | 8750 | 0.0005 | - |
385
- | 2.5157 | 8800 | 0.0004 | - |
386
- | 2.5300 | 8850 | 0.0004 | - |
387
- | 2.5443 | 8900 | 0.0005 | - |
388
- | 2.5586 | 8950 | 0.0005 | - |
389
- | 2.5729 | 9000 | 0.0005 | - |
390
- | 2.5872 | 9050 | 0.0004 | - |
391
- | 2.6015 | 9100 | 0.0005 | - |
392
- | 2.6158 | 9150 | 0.0004 | - |
393
- | 2.6301 | 9200 | 0.0011 | - |
394
- | 2.6444 | 9250 | 0.0004 | - |
395
- | 2.6587 | 9300 | 0.0005 | - |
396
- | 2.6730 | 9350 | 0.0006 | - |
397
- | 2.6872 | 9400 | 0.0005 | - |
398
- | 2.7015 | 9450 | 0.0005 | - |
399
- | 2.7158 | 9500 | 0.0004 | - |
400
- | 2.7301 | 9550 | 0.0006 | - |
401
- | 2.7444 | 9600 | 0.0004 | - |
402
- | 2.7587 | 9650 | 0.0005 | - |
403
- | 2.7730 | 9700 | 0.0005 | - |
404
- | 2.7873 | 9750 | 0.0005 | - |
405
- | 2.8016 | 9800 | 0.0004 | - |
406
- | 2.8159 | 9850 | 0.0004 | - |
407
- | 2.8302 | 9900 | 0.0004 | - |
408
- | 2.8445 | 9950 | 0.0005 | - |
409
- | 2.8588 | 10000 | 0.0004 | - |
410
- | 2.8731 | 10050 | 0.0004 | - |
411
- | 2.8874 | 10100 | 0.0005 | - |
412
- | 2.9017 | 10150 | 0.0004 | - |
413
- | 2.9160 | 10200 | 0.0004 | - |
414
- | 2.9302 | 10250 | 0.0005 | - |
415
- | 2.9445 | 10300 | 0.0004 | - |
416
- | 2.9588 | 10350 | 0.0005 | - |
417
- | 2.9731 | 10400 | 0.0004 | - |
418
- | 2.9874 | 10450 | 0.0005 | - |
419
- | 3.0 | 10494 | - | 0.0096 |
 
 
 
 
 
 
 
420
 
421
  ### Framework Versions
422
  - Python: 3.12.10
 
5
  - text-classification
6
  - generated_from_setfit_trainer
7
  widget:
8
+ - text: do I still need to take my medicine?
9
+ - text: send me another verification email
10
+ - text: at what time do I have dinner
11
+ - text: is it safe to take my insulin if my sugar is normal?
12
+ - text: turn on AR guides for test sessions
13
  metrics:
14
  - accuracy
15
  pipeline_tag: text-classification
 
17
  inference: true
18
  ---
19
 
20
+ ## Ausa Hub — intent router
21
+
22
+ On-device SetFit intent classifier for the **Ausa Hub** local voice assistant
23
+ (see `engine/router_config.py`). Maps a patient utterance to one of **39 intents**
24
+ (appointments, routines, profiles, tests/vitals, messages, meal_time, symptoms,
25
+ summary, settings, and the `system.*` catch-alls). Runs as ONNX on-device and
26
+ also seeds the cloud LLM router's priors.
27
+
28
+ - **Intents:** 39
29
+ - **Training examples:** ~1699
30
+ - **Last updated:** 2026-06-25
31
+ - **Recent change:** conversation-meta / recall utterances ("what did I just
32
+ say", "what did I tell you my dog's name was", "remind me what I said") and
33
+ general-knowledge questions now classify as `system.oos` so they reach the
34
+ conversational assistant instead of a records intent.
35
+
36
  # SetFit
37
 
38
  This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
 
61
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
62
 
63
  ### Model Labels
64
+ | Label | Examples |
65
+ |:------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
66
+ | messages.send | <ul><li>'should I take a half dose of my statin?'</li><li>'message aakash hi'</li><li>'Should I be worried about this reading?'</li></ul> |
67
+ | appointments.read | <ul><li>'what appointments do I have this week'</li><li>'are there any earlier slots'</li><li>'what times are available tomorrow'</li></ul> |
68
+ | profiles.care_team | <ul><li>'does my doctor have any openings this week'</li><li>"what is my doctor's phone number"</li><li>'list my assigned medical professionals'</li></ul> |
69
+ | system.oos | <ul><li>'twice a day'</li><li>'update that one'</li><li>'what is your name'</li></ul> |
70
+ | routines.update | <ul><li>'update my meditation routine duration from 10 to 20 minutes'</li><li>'update my yoga routine to include weekends'</li><li>"I need to push my morning jog routine back by an hour to 7 AM because it's getting too cold."</li></ul> |
71
+ | summary.read | <ul><li>"what's happening for me today"</li><li>'fill me in on my day'</li><li>'give me a rundown of my day'</li></ul> |
72
+ | tests.take_test | <ul><li>'let me do a temperature check'</li><li>'glucose test'</li><li>'begin a health test right now'</li></ul> |
73
+ | tests.test_history | <ul><li>'are my vitals normal'</li><li>'show me my vitals'</li><li>'is my blood sugar too high'</li></ul> |
74
+ | profiles.family.update | <ul><li>"extend my mother's data sharing for 90 more days"</li><li>"extend my son's data sharing access for another 30 days"</li><li>"update my brother-in-law's sharing to permanent"</li></ul> |
75
+ | symptoms.report | <ul><li>"I'm getting these chest palpitations"</li><li>'I have a migraine'</li><li>"i'm not well"</li></ul> |
76
+ | meal_time.update | <ul><li>'push my breakfast back to 7:30'</li><li>'set my dinner meal time to 8pm'</li><li>'set breakfast to 8am'</li></ul> |
77
+ | meal_time.read | <ul><li>'do I have a meal coming up'</li><li>'what time do I have my meals'</li><li>'what time is breakfast'</li></ul> |
78
+ | profiles.personal_info.picture | <ul><li>'choose a different photo for my profile'</li><li>'I need a new avatar picture'</li><li>'Let me snap a quick selfie to replace my current avatar.'</li></ul> |
79
+ | settings.calls | <ul><li>'warn me about poor connection during calls'</li><li>'enable background blur and closed captions'</li><li>'keep camera off when joining a call'</li></ul> |
80
+ | routines.read | <ul><li>'show my routines for today'</li><li>'what is the schedule for my gym routine'</li><li>'cancel this, show my routine schedule'</li></ul> |
81
+ | profiles.read | <ul><li>'what conditions have been diagnosed for me'</li><li>'what is my medical history'</li><li>'what medical issues are on my record'</li></ul> |
82
+ | profiles.personal_info.read | <ul><li>'what email is on my account'</li><li>'show me my email address'</li><li>'what is my email'</li></ul> |
83
+ | profiles.allergies.update | <ul><li>'cancel, I want to update my allergy info'</li><li>'My reaction to dairy has gotten worse, change the severity to high and add stomach cramps.'</li><li>'update my cat allergy to include asthma symptoms'</li></ul> |
84
+ | appointments.create | <ul><li>'arrange a visit for next Friday I need a prescription refill'</li><li>'leave it, I want to schedule a doctor visit instead'</li><li>'I want to schedule a checkup'</li></ul> |
85
+ | profiles.update | <ul><li>'I need to change my primary care physician'</li><li>'update my emergency contact to my wife'</li><li>'update my insurance plan details'</li></ul> |
86
+ | routines.create | <ul><li>'set up a blood pressure testing routine'</li><li>'create a routine to take my aspirin 81mg every day after lunch'</li><li>'create a monthly routine for my doctor follow-up on the first Monday'</li></ul> |
87
+ | settings.wifi | <ul><li>'I need to switch wifi networks'</li><li>'configure my wifi settings'</li><li>'show me my saved wifi networks'</li></ul> |
88
+ | profiles.family.read | <ul><li>'list all family members connected'</li><li>'show me who I am sharing my health data with'</li><li>'show me my family'</li></ul> |
89
+ | profiles.family.create | <ul><li>'invite someone to my account'</li><li>'I want to invite my husband to my care circle so he can keep track of my appointments.'</li><li>'let someone see my health information'</li></ul> |
90
+ | messages.read | <ul><li>'show my messages'</li><li>'show me my recent messages'</li><li>'what did Dr. Chen say'</li></ul> |
91
+ | profiles.personal_info.verify_email | <ul><li>'validate my email with a code'</li><li>'verify my email address'</li><li>'resend the email confirmation link'</li></ul> |
92
+ | profiles.family.delete | <ul><li>'delete my sister from family sharing'</li><li>'delete my neighbor from my care circle'</li><li>'delete all sharing with my in-laws'</li></ul> |
93
+ | appointments.delete | <ul><li>'take my Thursday appointment off the calendar'</li><li>"Scrap the follow-up appointment with the dermatologist, I won't be needing it."</li><li>'cancel the telehealth call I scheduled'</li></ul> |
94
+ | routines.delete | <ul><li>'take off my insulin shot reminder'</li><li>'erase this routine from my list forever'</li><li>'I do not need this routine anymore'</li></ul> |
95
+ | settings.display | <ul><li>'what is my current brightness level'</li><li>'brighten the display'</li><li>'dim the screen brightness'</li></ul> |
96
+ | settings.notifications | <ul><li>'show me my current notification settings'</li><li>'I want to make sure I get pinged whenever my care team sends a message, turn those on.'</li><li>'disable care team notifications'</li></ul> |
97
+ | system.cancel | <ul><li>'disregard'</li><li>'can we just stop'</li><li>"let's not"</li></ul> |
98
+ | profiles.allergies.read | <ul><li>'check if I have a sulfa drug allergy'</li><li>'tell me about my allergy details'</li><li>'is there an egg allergy on my profile'</li></ul> |
99
+ | profiles.allergies.delete | <ul><li>'delete an old allergy that is not relevant'</li><li>'get rid of the egg allergy entry'</li><li>'delete the nickel allergy from my file'</li></ul> |
100
+ | settings.devices | <ul><li>'connect a health monitoring device'</li><li>'remove the old thermometer from my devices'</li><li>'is my blood pressure cuff connected'</li></ul> |
101
+ | profiles.personal_info.update | <ul><li>'update my last name to Johnson'</li><li>'nevermind, change my height'</li><li>'change my date of birth'</li></ul> |
102
+ | appointments.update | <ul><li>'reschedule my appointment from morning to afternoon'</li><li>'stop, can you change my appointment time instead'</li><li>'I need to change the time of my visit'</li></ul> |
103
+ | profiles.personal_info.verify_phone | <ul><li>'nevermind, verify my phone'</li><li>'I changed my number and need to verify it'</li><li>'trigger a new phone verification code'</li></ul> |
104
+ | profiles.allergies.create | <ul><li>'I found out I am allergic to something new'</li><li>'put down a soy allergy with mild stomach discomfort'</li><li>'add a penicillin allergy with severe severity I get hives and swelling and it started in childhood'</li></ul> |
105
 
106
  ## Uses
107
 
 
121
  # Download from the 🤗 Hub
122
  model = SetFitModel.from_pretrained("setfit_model_id")
123
  # Run inference
124
+ preds = model("at what time do I have dinner")
125
  ```
126
 
127
  <!--
 
153
  ### Training Set Metrics
154
  | Training set | Min | Median | Max |
155
  |:-------------|:----|:-------|:----|
156
+ | Word count | 1 | 6.2825 | 21 |
157
 
158
  | Label | Training Sample Count |
159
  |:------------------------------------|:----------------------|
 
166
  | messages.read | 39 |
167
  | messages.send | 109 |
168
  | profiles.allergies.create | 22 |
169
+ | profiles.allergies.delete | 26 |
170
  | profiles.allergies.read | 22 |
171
  | profiles.allergies.update | 22 |
172
  | profiles.care_team | 40 |
 
193
  | summary.read | 32 |
194
  | symptoms.report | 44 |
195
  | system.cancel | 27 |
196
+ | system.oos | 180 |
197
  | tests.take_test | 31 |
198
+ | tests.test_history | 76 |
199
 
200
  ### Training Hyperparameters
201
  - batch_size: (16, 16)
 
221
  | Epoch | Step | Training Loss | Validation Loss |
222
  |:------:|:-----:|:-------------:|:---------------:|
223
  | 0.0003 | 1 | 0.0006 | - |
224
+ | 0.0139 | 50 | 0.0042 | - |
225
+ | 0.0277 | 100 | 0.0011 | - |
226
+ | 0.0416 | 150 | 0.0009 | - |
227
+ | 0.0554 | 200 | 0.0004 | - |
228
+ | 0.0693 | 250 | 0.0005 | - |
229
+ | 0.0831 | 300 | 0.0004 | - |
230
+ | 0.0970 | 350 | 0.0005 | - |
231
+ | 0.1108 | 400 | 0.0004 | - |
232
+ | 0.1247 | 450 | 0.0004 | - |
233
+ | 0.1385 | 500 | 0.0005 | - |
234
+ | 0.1524 | 550 | 0.0004 | - |
235
+ | 0.1662 | 600 | 0.0004 | - |
236
+ | 0.1801 | 650 | 0.0004 | - |
237
+ | 0.1939 | 700 | 0.0004 | - |
238
+ | 0.2078 | 750 | 0.0004 | - |
239
+ | 0.2216 | 800 | 0.0004 | - |
240
+ | 0.2355 | 850 | 0.0005 | - |
241
+ | 0.2493 | 900 | 0.0005 | - |
242
+ | 0.2632 | 950 | 0.0005 | - |
243
+ | 0.2770 | 1000 | 0.0005 | - |
244
+ | 0.2909 | 1050 | 0.0005 | - |
245
+ | 0.3047 | 1100 | 0.0005 | - |
246
+ | 0.3186 | 1150 | 0.0005 | - |
247
+ | 0.3324 | 1200 | 0.0006 | - |
248
+ | 0.3463 | 1250 | 0.0005 | - |
249
+ | 0.3601 | 1300 | 0.0005 | - |
250
+ | 0.3740 | 1350 | 0.0005 | - |
251
+ | 0.3878 | 1400 | 0.0032 | - |
252
+ | 0.4017 | 1450 | 0.0006 | - |
253
+ | 0.4155 | 1500 | 0.0006 | - |
254
+ | 0.4294 | 1550 | 0.0005 | - |
255
+ | 0.4432 | 1600 | 0.0005 | - |
256
+ | 0.4571 | 1650 | 0.0005 | - |
257
+ | 0.4709 | 1700 | 0.0007 | - |
258
+ | 0.4848 | 1750 | 0.0005 | - |
259
+ | 0.4986 | 1800 | 0.0005 | - |
260
+ | 0.5125 | 1850 | 0.0005 | - |
261
+ | 0.5263 | 1900 | 0.0006 | - |
262
+ | 0.5402 | 1950 | 0.0005 | - |
263
+ | 0.5540 | 2000 | 0.0005 | - |
264
+ | 0.5679 | 2050 | 0.0005 | - |
265
+ | 0.5817 | 2100 | 0.0005 | - |
266
+ | 0.5956 | 2150 | 0.0006 | - |
267
+ | 0.6094 | 2200 | 0.0005 | - |
268
+ | 0.6233 | 2250 | 0.0004 | - |
269
+ | 0.6371 | 2300 | 0.0005 | - |
270
+ | 0.6510 | 2350 | 0.0005 | - |
271
+ | 0.6648 | 2400 | 0.0005 | - |
272
+ | 0.6787 | 2450 | 0.0005 | - |
273
+ | 0.6925 | 2500 | 0.0005 | - |
274
+ | 0.7064 | 2550 | 0.0005 | - |
275
+ | 0.7202 | 2600 | 0.0004 | - |
276
+ | 0.7341 | 2650 | 0.0013 | - |
277
+ | 0.7479 | 2700 | 0.0005 | - |
278
+ | 0.7618 | 2750 | 0.0005 | - |
279
+ | 0.7756 | 2800 | 0.0005 | - |
280
+ | 0.7895 | 2850 | 0.0004 | - |
281
+ | 0.8033 | 2900 | 0.0005 | - |
282
+ | 0.8172 | 2950 | 0.0005 | - |
283
+ | 0.8310 | 3000 | 0.0004 | - |
284
+ | 0.8449 | 3050 | 0.0005 | - |
285
+ | 0.8587 | 3100 | 0.0007 | - |
286
+ | 0.8726 | 3150 | 0.0004 | - |
287
+ | 0.8864 | 3200 | 0.0005 | - |
288
+ | 0.9003 | 3250 | 0.0005 | - |
289
+ | 0.9141 | 3300 | 0.0005 | - |
290
+ | 0.9280 | 3350 | 0.0004 | - |
291
+ | 0.9418 | 3400 | 0.0006 | - |
292
+ | 0.9557 | 3450 | 0.0004 | - |
293
+ | 0.9695 | 3500 | 0.0005 | - |
294
+ | 0.9834 | 3550 | 0.0004 | - |
295
+ | 0.9972 | 3600 | 0.0008 | - |
296
+ | 1.0 | 3610 | - | 0.0001 |
297
+ | 1.0111 | 3650 | 0.0004 | - |
298
+ | 1.0249 | 3700 | 0.0006 | - |
299
+ | 1.0388 | 3750 | 0.0005 | - |
300
+ | 1.0526 | 3800 | 0.0004 | - |
301
+ | 1.0665 | 3850 | 0.0007 | - |
302
+ | 1.0803 | 3900 | 0.0005 | - |
303
+ | 1.0942 | 3950 | 0.0004 | - |
304
+ | 1.1080 | 4000 | 0.0005 | - |
305
+ | 1.1219 | 4050 | 0.0004 | - |
306
+ | 1.1357 | 4100 | 0.0005 | - |
307
+ | 1.1496 | 4150 | 0.0004 | - |
308
+ | 1.1634 | 4200 | 0.0008 | - |
309
+ | 1.1773 | 4250 | 0.0004 | - |
310
+ | 1.1911 | 4300 | 0.0004 | - |
311
+ | 1.2050 | 4350 | 0.0004 | - |
312
+ | 1.2188 | 4400 | 0.0004 | - |
313
+ | 1.2327 | 4450 | 0.0004 | - |
314
+ | 1.2465 | 4500 | 0.0005 | - |
315
+ | 1.2604 | 4550 | 0.0004 | - |
316
+ | 1.2742 | 4600 | 0.0004 | - |
317
+ | 1.2881 | 4650 | 0.0005 | - |
318
+ | 1.3019 | 4700 | 0.0004 | - |
319
+ | 1.3158 | 4750 | 0.0005 | - |
320
+ | 1.3296 | 4800 | 0.0004 | - |
321
+ | 1.3435 | 4850 | 0.0004 | - |
322
+ | 1.3573 | 4900 | 0.0004 | - |
323
+ | 1.3712 | 4950 | 0.0004 | - |
324
+ | 1.3850 | 5000 | 0.0005 | - |
325
+ | 1.3989 | 5050 | 0.0004 | - |
326
+ | 1.4127 | 5100 | 0.0004 | - |
327
+ | 1.4266 | 5150 | 0.0004 | - |
328
+ | 1.4404 | 5200 | 0.0007 | - |
329
+ | 1.4543 | 5250 | 0.0004 | - |
330
+ | 1.4681 | 5300 | 0.0004 | - |
331
+ | 1.4820 | 5350 | 0.0004 | - |
332
+ | 1.4958 | 5400 | 0.0005 | - |
333
+ | 1.5097 | 5450 | 0.0004 | - |
334
+ | 1.5235 | 5500 | 0.0005 | - |
335
+ | 1.5374 | 5550 | 0.0004 | - |
336
+ | 1.5512 | 5600 | 0.0005 | - |
337
+ | 1.5651 | 5650 | 0.0004 | - |
338
+ | 1.5789 | 5700 | 0.0004 | - |
339
+ | 1.5928 | 5750 | 0.0004 | - |
340
+ | 1.6066 | 5800 | 0.0004 | - |
341
+ | 1.6205 | 5850 | 0.0004 | - |
342
+ | 1.6343 | 5900 | 0.0004 | - |
343
+ | 1.6482 | 5950 | 0.0004 | - |
344
+ | 1.6620 | 6000 | 0.0005 | - |
345
+ | 1.6759 | 6050 | 0.0005 | - |
346
+ | 1.6898 | 6100 | 0.0004 | - |
347
+ | 1.7036 | 6150 | 0.0012 | - |
348
+ | 1.7175 | 6200 | 0.0007 | - |
349
+ | 1.7313 | 6250 | 0.0006 | - |
350
+ | 1.7452 | 6300 | 0.0015 | - |
351
+ | 1.7590 | 6350 | 0.0005 | - |
352
+ | 1.7729 | 6400 | 0.0004 | - |
353
+ | 1.7867 | 6450 | 0.0005 | - |
354
+ | 1.8006 | 6500 | 0.0017 | - |
355
+ | 1.8144 | 6550 | 0.0004 | - |
356
+ | 1.8283 | 6600 | 0.0004 | - |
357
+ | 1.8421 | 6650 | 0.0004 | - |
358
+ | 1.8560 | 6700 | 0.0004 | - |
359
+ | 1.8698 | 6750 | 0.0004 | - |
360
+ | 1.8837 | 6800 | 0.0004 | - |
361
+ | 1.8975 | 6850 | 0.0004 | - |
362
+ | 1.9114 | 6900 | 0.0004 | - |
363
+ | 1.9252 | 6950 | 0.0004 | - |
364
+ | 1.9391 | 7000 | 0.0004 | - |
365
+ | 1.9529 | 7050 | 0.0004 | - |
366
+ | 1.9668 | 7100 | 0.0004 | - |
367
+ | 1.9806 | 7150 | 0.0004 | - |
368
+ | 1.9945 | 7200 | 0.0004 | - |
369
+ | 2.0 | 7220 | - | 0.0001 |
370
+ | 2.0083 | 7250 | 0.0005 | - |
371
+ | 2.0222 | 7300 | 0.0004 | - |
372
+ | 2.0360 | 7350 | 0.0006 | - |
373
+ | 2.0499 | 7400 | 0.0004 | - |
374
+ | 2.0637 | 7450 | 0.0004 | - |
375
+ | 2.0776 | 7500 | 0.0003 | - |
376
+ | 2.0914 | 7550 | 0.0004 | - |
377
+ | 2.1053 | 7600 | 0.0005 | - |
378
+ | 2.1191 | 7650 | 0.0004 | - |
379
+ | 2.1330 | 7700 | 0.0004 | - |
380
+ | 2.1468 | 7750 | 0.0004 | - |
381
+ | 2.1607 | 7800 | 0.0004 | - |
382
+ | 2.1745 | 7850 | 0.0004 | - |
383
+ | 2.1884 | 7900 | 0.0004 | - |
384
+ | 2.2022 | 7950 | 0.0004 | - |
385
+ | 2.2161 | 8000 | 0.0004 | - |
386
+ | 2.2299 | 8050 | 0.0005 | - |
387
+ | 2.2438 | 8100 | 0.0004 | - |
388
+ | 2.2576 | 8150 | 0.0004 | - |
389
+ | 2.2715 | 8200 | 0.0004 | - |
390
+ | 2.2853 | 8250 | 0.0004 | - |
391
+ | 2.2992 | 8300 | 0.0004 | - |
392
+ | 2.3130 | 8350 | 0.0004 | - |
393
+ | 2.3269 | 8400 | 0.0004 | - |
394
+ | 2.3407 | 8450 | 0.0004 | - |
395
+ | 2.3546 | 8500 | 0.0004 | - |
396
+ | 2.3684 | 8550 | 0.0004 | - |
397
+ | 2.3823 | 8600 | 0.0007 | - |
398
+ | 2.3961 | 8650 | 0.0004 | - |
399
+ | 2.4100 | 8700 | 0.0004 | - |
400
+ | 2.4238 | 8750 | 0.0004 | - |
401
+ | 2.4377 | 8800 | 0.0004 | - |
402
+ | 2.4515 | 8850 | 0.0003 | - |
403
+ | 2.4654 | 8900 | 0.0003 | - |
404
+ | 2.4792 | 8950 | 0.0004 | - |
405
+ | 2.4931 | 9000 | 0.0004 | - |
406
+ | 2.5069 | 9050 | 0.0004 | - |
407
+ | 2.5208 | 9100 | 0.0006 | - |
408
+ | 2.5346 | 9150 | 0.0004 | - |
409
+ | 2.5485 | 9200 | 0.0004 | - |
410
+ | 2.5623 | 9250 | 0.0004 | - |
411
+ | 2.5762 | 9300 | 0.0004 | - |
412
+ | 2.5900 | 9350 | 0.0003 | - |
413
+ | 2.6039 | 9400 | 0.0004 | - |
414
+ | 2.6177 | 9450 | 0.0003 | - |
415
+ | 2.6316 | 9500 | 0.0004 | - |
416
+ | 2.6454 | 9550 | 0.0003 | - |
417
+ | 2.6593 | 9600 | 0.0004 | - |
418
+ | 2.6731 | 9650 | 0.0004 | - |
419
+ | 2.6870 | 9700 | 0.0004 | - |
420
+ | 2.7008 | 9750 | 0.0003 | - |
421
+ | 2.7147 | 9800 | 0.0003 | - |
422
+ | 2.7285 | 9850 | 0.0003 | - |
423
+ | 2.7424 | 9900 | 0.0004 | - |
424
+ | 2.7562 | 9950 | 0.0003 | - |
425
+ | 2.7701 | 10000 | 0.0003 | - |
426
+ | 2.7839 | 10050 | 0.0005 | - |
427
+ | 2.7978 | 10100 | 0.0004 | - |
428
+ | 2.8116 | 10150 | 0.0003 | - |
429
+ | 2.8255 | 10200 | 0.0003 | - |
430
+ | 2.8393 | 10250 | 0.0003 | - |
431
+ | 2.8532 | 10300 | 0.0003 | - |
432
+ | 2.8670 | 10350 | 0.0004 | - |
433
+ | 2.8809 | 10400 | 0.0003 | - |
434
+ | 2.8947 | 10450 | 0.0004 | - |
435
+ | 2.9086 | 10500 | 0.0003 | - |
436
+ | 2.9224 | 10550 | 0.0004 | - |
437
+ | 2.9363 | 10600 | 0.0004 | - |
438
+ | 2.9501 | 10650 | 0.0004 | - |
439
+ | 2.9640 | 10700 | 0.0003 | - |
440
+ | 2.9778 | 10750 | 0.0003 | - |
441
+ | 2.9917 | 10800 | 0.0003 | - |
442
+ | 3.0 | 10830 | - | 0.0000 |
443
 
444
  ### Framework Versions
445
  - Python: 3.12.10
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8fdff30ea7426c80f28018286aeedf357dc1693b1762cd6f7991d7aab6382058
3
  size 133462128
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c2e51ce6205f8ceddc4a2be9267a16bd58f126f60b573ec11dbce7045f558bca
3
  size 133462128
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:330feef7f0e0c66ef73f62dbc5ea120108d0fd439a6d8ced02e660a5cfca8dda
3
  size 121279
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58f05a320aa840e0c5b06e6d3c2ad3f34ccbe6dcb3074b3b6857afa974c1a750
3
  size 121279