Resume screening reference bundle

This file is a reference bundle for scoring uploaded resumes. It contains derived numbers from 2,481 Kaggle resumes and never contains resume text.

Bundle contents

  • tf_data: (804340,)
  • tf_indices: (804340,)
  • tf_indptr: (2482,)
  • tf_shape: (2,)
  • vocabulary: (39935,)
  • idf: (39935,)
  • doc_len: (2481,)
  • ref_embeddings: (2481, 384)
  • ref_ids: (2481,)
  • ref_categories: (2481,)
  • ref_skill_sets_json: ()
  • skill_aliases_json: ()
  • config_json: ()

Use

Load the file with numpy.load(..., allow_pickle=False) and pass the resulting bundle through machine_learning.score_uploaded.

The hybrid weights are {"bm25": 0.1, "embedding": 0.8, "skills": 0.1}. The embedding model is sentence-transformers/all-MiniLM-L6-v2 at revision 1110a243fdf4706b3f48f1d95db1a4f5529b4d41.

Test metrics

The following values are read from outputs/metrics.csv:

method,P@10,MRR,NDCG@10,NDCG@10 graded
TF IDF,0.457,0.575,0.452,0.535
BM25,0.466,0.609,0.465,0.531
minilm full,0.476,0.617,0.473,0.562
minilm chunked,0.454,0.584,0.445,0.53
bge_small full,0.503,0.627,0.494,0.557
bge_small chunked,0.451,0.627,0.451,0.529
Skills only,0.182,0.173,0.16,0.243
Hybrid,0.486,0.596,0.48,0.561

Confidence intervals are read from outputs/readme_results.md:

### Held out test results

| Method | P@10 | MRR | NDCG@10 | NDCG@10 graded |
| --- | --- | --- | --- | --- |
| TF IDF | 0.457 | 0.575 | 0.452 | 0.535 |
| BM25 | 0.466 | 0.609 | 0.465 | 0.531 |
| minilm full | 0.476 | 0.617 | 0.473 | 0.562 |
| minilm chunked | 0.454 | 0.584 | 0.445 | 0.53 |
| bge_small full | 0.503 | 0.627 | 0.494 | 0.557 |
| bge_small chunked | 0.451 | 0.627 | 0.451 | 0.529 |
| Skills only | 0.182 | 0.173 | 0.16 | 0.243 |
| Hybrid | 0.486 | 0.596 | 0.48 | 0.561 |

### Graded NDCG@10 with 95% bootstrap confidence intervals

| method | NDCG@10 graded | CI low | CI high |
| --- | --- | --- | --- |
| TF IDF | 0.535 | 0.459 | 0.613 |
| BM25 | 0.531 | 0.46 | 0.602 |
| Best embedding (minilm full) | 0.562 | 0.481 | 0.637 |
| Hybrid | 0.561 | 0.479 | 0.637 |

* Hybrid minus TF IDF: 0.026 (95% CI -0.015 to 0.068)
* Hybrid minus BM25: 0.031 (95% CI -0.019 to 0.079)

Best embedding variant (chosen on tuning queries): minilm full
Hybrid weights: {'bm25': 0.1, 'embedding': 0.8, 'skills': 0.1}

Pinned library versions are read from requirements.txt:

fastapi==0.163.2
uvicorn==0.54.0
pydantic==2.14.3
python_multipart==0.0.34
pypdf==7.1.0
numpy==2.5.5
scipy==1.18.1
scikit_learn==1.9.1
spacy==3.8.18
sentence_transformers==6.3.0
torch==2.15.0
huggingface_hub==2.4.0

Limitations

Category labels are a proxy, the ADVOCATE category was excluded, and job descriptions are synthetic. Scores are relative to the reference pool. Source dataset licences were not verified.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support