commure-smislam/email-classification-validated

Email Classification Model - Pipeline Validated

A production-ready email classification model for healthcare communications.

Performance Metrics

Model Performance:

• Subcategory F1 (macro): 25.6%
• Category Accuracy: 61.6%
• Top-2 Accuracy: 0.0%

Pipeline Validated:

• Real data loading (44,602 emails)
• Stratified splitting
• Fast training convergence
• Dual-head classification
• HuggingFace deployment

Categories (19)

air.calendar, air.patient_info, air.practice, air.product, appointments, athelas_ehr, athelas_scribe, business_update, denials, eligibility...

Subcategories (86)

Top categories: appointment.incorrect_info, appointment.missing, appointment.reminder, business_update.additional_meeting_request, business_update.banking_update, business_update.billing_process_update, business_update.custom_report, business_update.edi_era_enrollments, business_update.facility_update, business_update.fee_schedule_update

Usage

import requests

API_URL = "https://your-endpoint.aws.endpoints.huggingface.cloud"
headers = {"Authorization": "Bearer YOUR_TOKEN"}

response = requests.post(API_URL, headers=headers, json={
    "inputs": "Patient appointment confirmation for tomorrow"
})

result = response.json()
# {
#   "category": {"label": "appointments", "confidence": 0.95},
#   "subcategory": {"label": "appointments.confirmation", "confidence": 0.92}
# }

Model Details

• Base Model: distilbert-base-uncased
• Training Samples: 37,911
• Test Samples: 6,691
• Parameters: 66,443,625
• Max Length: 256

Files

• model_checkpoint.pt: Complete model checkpoint
• tokenizer/: HuggingFace tokenizer
• handler.py: Inference endpoint handler