README.md

---
license: apache-2.0
base_model: AlexWortega/openjev
library_name: peft
language: [en]
tags: [skills-assessment, resume, cv, nli, lora, classification, hr-tech]
datasets: [niko0xdev/skilljev-data]
pipeline_tag: text-classification
---
# SkillJev: CV-evidence → skill / evidence / proficiency classifier

SkillJev scores how well candidate evidence (CV excerpts, interview notes) supports a single job requirement. It has three heads:

| Head | Labels |
|---|---|
| **skill group** | `lang_core, api_design, data_storage, system_design, cloud_devops, reliability_obs, security, testing_quality, frontend, leadership` |
| **evidence** | `supported, partial, contradicted, insufficient` |
| **proficiency** | `0, 1, 2, 3, 4, 5, unknown` (**unknown ≠ 0**: level 0 is used only when the evidence explicitly says the candidate has no experience) |

- **Base model:** [`AlexWortega/openjev`](https://huggingface.co/AlexWortega/openjev), subfolder `qwen3.5-4b-nli-v5` (OpenJev 4B, Qwen3.5 NLI).
- **Adapter:** LoRA r=16, α=32, all linear layers.
- **Heads:** 3 linear heads on the last-token hidden state (`heads.pt`).
- **Training:** 1 epoch-slice, bf16, lr 1e-4, batch 4, max length 192, 500 steps on Colab L4.
- **Training data:** [`niko0xdev/skilljev-data`](https://huggingface.co/datasets/niko0xdev/skilljev-data) (v008).
- **Labeling standard:** `RUBRIC.md`, including the *Real-CV labeling convention v0.2*.

## Results: locked held-out set (n = 90, never trained on)
The held-out set has 70 synthetic samples built from **held-out-only templates** and 20 human/agent-reviewed real Djinni CV excerpts.

| Model | skill | evidence | prof exact | prof ±1 | unknown→0 errors | ECE (ev / prof) |
|---|---|---|---|---|---|---|
| tasksource/ModernBERT-base-nli, zero-shot | 73.3 | 56.7 | 58.9 | 63.3 | 0 | – |
| OpenJev 4B, zero-shot | 88.9 | 68.9 | 71.1 | 80.0 | 0 | – |
| **SkillJev (this checkpoint: v008, step 500)** | **100.0** | **97.8** | **95.6** | 96.7 | 0 | 0.023 / 0.033 |
| SkillJev v008, step 1000 (dev-selected) | 100.0 | 97.8 | 94.4 | 98.9 | 0 | 0.024 / 0.038 |

ECE is measured after temperature scaling fit on the dev split (T: skill 0.5, evidence 2.4, proficiency 2.8).

> **Caveat:** step 500 was chosen because of its held-out score. The checkpoint picked purely on dev (step 1000) reaches 94.4% proficiency exact. The 1.2-point gap is one sample out of 90, so treat ~94–96% as the realistic range. The held-out set is small (n=90), and every number carries roughly ±3–5 points of sampling noise.

## Usage
```python
# pip install "transformers>=4.57" peft accelerate  (GPU with ~12 GB)
# see inference.py in this repo
from inference import predict
predict("Strong PostgreSQL skills", [("cv","experience","Partitioned a 3TB PostgreSQL table and tuned indexes, cutting queries from 40s to 1s.")])
# {'skill_group': {'label': 'data_storage', ...}, 'evidence': {'label': 'supported', ...}, 'proficiency': {'label': '4', ...}}
```
Input format: `Requirement: <text>\nEvidence:\n[<source_type>/<section>] <text>` (one line per evidence item, or `(no evidence)`).

## Intended use & limitations
- This is a **decision-support** tool for structuring CV screening. It is not an automated hiring decision. Keep a human in the loop and route low-confidence (<0.7), `contradicted`, or level ≥4 CV-only predictions to review.
- Coverage: English, IT/software roles, 55 skills in 10 groups (`TAXONOMY.md`). Other domains or languages are out of distribution.
- Most training data is template-generated synthetic text. The real-CV portion is about 1.2k agent-reviewed excerpts plus about 5.7k rule-labelled (silver) excerpts, so expect lower accuracy on unusual phrasing.
- Proficiency is judged from a single excerpt. It reflects the evidence given, not the person's true skill. CV self-reports are not verified.
- Possible biases come from the Djinni (Ukrainian job market) CV distribution and from the labeling convention itself.

## MLX / Apple silicon
The release is standard PEFT safetensors plus a small PyTorch `heads.pt`. To run on MLX: merge the LoRA into the base model (`peft` `merge_and_unload()`), convert with `mlx_lm.convert`, and port the three linear heads (weights in `heads.pt`) to `mlx.nn.Linear`. No official MLX build is provided.

## License & attribution
- The adapter is released under Apache-2.0. The base model's own license also applies.
- Real CV excerpts come from [lang-uk/recruitment-dataset-candidate-profiles-english](https://huggingface.co/datasets/lang-uk/recruitment-dataset-candidate-profiles-english) (MIT).