| --- |
| language: |
| - en |
| - fr |
| - de |
| license: mit |
| library_name: sentence-transformers |
| base_model: intfloat/multilingual-e5-small |
| pipeline_tag: sentence-similarity |
| tags: |
| - sentence-transformers |
| - sentence-similarity |
| - feature-extraction |
| - job-titles |
| - retrieval |
| - contrastive-learning |
| - matryoshka |
| - cross-lingual |
| - esco |
| - onet |
| --- |
| |
| # job-title-normalizer-e5-small |
|
|
| The **lightweight variant** of |
| [job-title-normalizer-e5-base](https://huggingface.co/Misbahuddin/job-title-normalizer-e5-base): |
| a 118M-param sentence encoder fine-tuned to **normalize messy, multilingual job titles to a |
| canonical occupation taxonomy** (ESCO + O*NET), framed as retrieval over 4,055 canonical |
| occupations. ~2.5× smaller and faster than the e5-base variant at a modest accuracy cost — |
| the right choice for CPU serving and bulk backfills. |
| |
| **Live demo:** <https://job-title-normalizer-525186107937.us-central1.run.app> (Cloud Run; may cold-start ~1 min) |
| |
| ## Results |
| |
| Held-out test set of **15,248 real ESCO/O*NET titles**; split **by occupation** (zero |
| train/test occupation overlap, asserted at build time). |
| |
| | Slice | Method | Recall@1 | Recall@5 | Recall@10 | MRR | |
| |---|---|---|---|---|---| |
| | Overall | BM25 (lexical) | 0.095 | 0.163 | 0.183 | 0.124 | |
| | Overall | zero-shot e5-small | 0.226 | 0.371 | 0.437 | 0.286 | |
| | Overall | **this model** | **0.367** | **0.539** | **0.601** | **0.441** | |
| | FR→EN | **this model** | **0.525** | **0.737** | **0.799** | **0.618** | |
| | DE→EN | **this model** | **0.523** | **0.745** | **0.815** | **0.619** | |
| |
| BM25 scores **MRR 0.034** cross-lingually; this model reaches **0.619**. For the best |
| accuracy, use the [e5-base variant](https://huggingface.co/Misbahuddin/job-title-normalizer-e5-base) |
| (overall MRR 0.463 vs 0.441). |
| |
| ## Training |
| |
| - **Base:** [`intfloat/multilingual-e5-small`](https://huggingface.co/intfloat/multilingual-e5-small) (mean pooling, `query:`/`passage:` prefixes, 384-dim). |
| - **Objective:** in-batch-negatives InfoNCE (`MultipleNegativesRankingLoss`, scale 20), wrapped in **MatryoshkaLoss** (dims 384/256/128/64). |
| - **Data:** 160,240 positive pairs (synonyms, alternate titles, cross-lingual label pairs) from ESCO (EN/FR/DE) + O*NET alternate titles; ≤50 pairs per occupation. |
| - **Setup:** 2 epochs, batch 128, lr 2e-5, warmup 10%, max_seq_len 64, fp16, `NoDuplicatesDataLoader`. Trained on an RTX 4060 Laptop (8 GB) in ~8 min. |
|
|
| ## How to use |
|
|
| The e5 family needs **asymmetric prefixes**: titles as `query: …`, canonical labels as |
| `passage: …`. Forgetting them silently degrades accuracy. |
|
|
| ```python |
| from sentence_transformers import SentenceTransformer |
| from sentence_transformers.util import cos_sim |
| |
| model = SentenceTransformer("Misbahuddin/job-title-normalizer-e5-small") |
| |
| canonicals = [ |
| "passage: software developer", |
| "passage: data scientist", |
| "passage: nurse responsible for general care", |
| ] |
| query = "query: Krankenpfleger" # German → English canonical |
| |
| q = model.encode(query, normalize_embeddings=True) |
| c = model.encode(canonicals, normalize_embeddings=True) |
| scores = cos_sim(q, c)[0] |
| print(canonicals[int(scores.argmax())], float(scores.max())) |
| ``` |
|
|
| Vectors are L2-normalized (cosine == dot product) — use FAISS `IndexFlatIP` for production. |
| Matryoshka truncation to 256/128/64 dims works after re-normalizing. **Calibrate an |
| abstention threshold**: out-of-taxonomy queries return low-score nearest neighbours rather |
| than failing. |
|
|
| ## Intended use & limitations |
|
|
| - **In scope:** normalizing titles from resumes, postings, CRM/ATS/HRIS records to occupation |
| IDs; semantic occupation search; FR/DE→EN cross-lingual lookup. |
| - **Not a hiring/screening/compensation decision system.** Keep a human in the loop for |
| consequential uses. |
| - **Coverage bounded by ESCO + O*NET**; en/fr/de verified only. Short-text regime (≤64 tokens). |
| |
| ## Acknowledgements |
| |
| - **ESCO** — © European Union; reused under Commission Decision 2011/833/EU. <https://esco.ec.europa.eu/> |
| - **O*NET** — by the National Center for O*NET Development for USDOL/ETA; CC BY 4.0. <https://www.onetcenter.org/> |
| |
| > O*NET® is a trademark of USDOL/ETA. This model was produced using O*NET data but is not |
| > endorsed by USDOL/ETA. ESCO is a service of the European Commission; this model is not |
| > endorsed by the Commission. |
| |
