RyzenShivansh/startup-email-lead-classifier

Startup service-inquiry email classifier

distilbert/distilroberta-base fine-tuned to classify purchase-oriented service inquiries (service_inquiry) versus other incoming email (not_service_inquiry). Production serving uses the included ONNX INT8 model and validation-selected routing thresholds.

Evaluation

Held-out grouped test metrics at threshold 0.1419:

precision	recall	F1	average precision
1.0000	0.9825	0.9912	1.0000

These scores only describe the dataset recorded in evaluation/. Synthetic/template examples are useful for bootstrapping, but do not establish accuracy on a new company's live inbox.

Intended use and limits

• English business email triage; max token length: 384.
• Human review is required for the uncertainty band and high-impact decisions.
• Re-evaluate on a frozen, manually labeled sample of the target inbox before production.
• Do not use this model for employment, credit, medical, or other consequential decisions.
• Email text can contain sensitive data; the API does not log request bodies.

Training provenance

• Base model: distilbert/distilroberta-base (MIT licensed)
• Seed: 42
• Dataset rows after cleaning: 1371
• Labels: 0=not_service_inquiry, 1=service_inquiry

Final ONNX INT8 grouped-test results

precision	recall	F1	average precision	automatic-lead precision	auto-rejected lead rate
1.0000	0.9649	0.9821	1.0000	1.0000	0.0000

INT8 size: 78.763 MB (74.86% smaller than FP32 ONNX).