README.md

---
title: ColdScore
emoji: 🧲
colorFrom: indigo
colorTo: gray
sdk: gradio
app_file: app.py
pinned: false
license: mit
short_description: Open-weights cold-email deliverability scoring on ModernBERT
---

# ColdScore

**Will this cold email reach the inbox — or the spam folder?**

A [ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) model (Apache-2.0,
149M params) fine-tuned on a corpus of 100 hand-labelled cold emails. Open weights, MIT
licence, runs on CPU.

Model: [`Ashsinha1/coldscore-modernbert`](https://huggingface.co/Ashsinha1/coldscore-modernbert)

## Why an open model instead of a frontier API

A frontier model would score these emails better. That isn't the point.

Scoring a cold email means handing over a prospect's name, their email address, and your
pitch — customer data. Sending that to a third-party model provider is a decision that
shows up in your customers' security review.

This model doesn't force that decision. The weights are public, inference is a CPU process,
and it runs inside your own network in four lines:

```python
from transformers import pipeline

clf = pipeline("text-classification", model="Ashsinha1/coldscore-modernbert")
clf({"text": "quick question about onboarding", "text_pair": "Hi Sarah, I noticed..."})
# [{'label': 'deliverable', 'score': 0.98}]
```

For the half of a pipeline that touches every contact in the database, "good and local"
beats "excellent and elsewhere".

## Why an encoder rather than an LLM

For binary text classification, a fine-tuned encoder wins on all three axes that matter:
accuracy, cost, and latency. ModernBERT is the current state of the art in that family —
trained on 2T tokens, 8,192-token context, so a whole email fits without truncation. At
149M parameters it scores an email in well under a second on CPU, where a 1.5B model takes
seconds per request.

## Training

- **Base:** `answerdotai/ModernBERT-base`
- **Data:** 100 hand-written, hand-labelled cold emails (50 deliverable / 50 spam-prone),
  expanded to 1,200 rows by filling `{first}` / `{company}` placeholders
- **Split:** grouped by seed email — every variant of an email lands on the same side, so
  near-duplicates cannot leak across the train/test boundary
- **Compute:** minutes on a single T4

## Limitations — read these before trusting a number

1. **Content only.** Real inbox placement is dominated by things the model cannot see:
   domain and IP reputation, SPF/DKIM/DMARC alignment, sending volume, list hygiene,
   complaint rates. A flawless email from a burned domain still lands in spam.
2. **The benchmark is easy.** The spam examples are unsubtle. A plain keyword blocklist
   already scores ~0.94 F1 on this corpus, so held-out numbers are an optimistic upper
   bound rather than a real-world accuracy estimate. The genuinely hard cases —
   blunt-but-legitimate outreach, polished promotional blasts — are barely represented.
3. **Labels are judgements, not measured outcomes.** Nobody sent these 100 emails and
   observed where they landed.
4. **Small corpus, one author's taste.** It likely over-rewards a short, curious,
   low-pressure style.
5. **English only.** Not a phishing or security detector — well-written phishing scores clean.

Treat it as a content lint, not a deliverability guarantee.

## Privacy

Text entered in the demo is processed in this Space's container and is not stored or logged.
If that still isn't good enough for your data — a reasonable position, and rather the point
of the project — clone the Space or run the model locally.

## Licence

MIT for this model and app. The base model is Apache-2.0.