README.md

---
license: apache-2.0
language:
- de
base_model: deepset/gbert-base
pipeline_tag: text-classification
library_name: transformers
tags:
- text-classification
- job-postings
- esco
- occupation-classification
- german
- stellen-atlas
datasets:
- mischeiwiller/german-job-postings
metrics:
- f1
- accuracy
model-index:
- name: jobbert-de
  results:
  - task:
      type: text-classification
      name: ESCO occupation classification (occupation-level)
    dataset:
      name: Stellen-Atlas gold (208 rows / 135 ESCO occupations)
      type: mischeiwiller/german-job-postings
      split: gold
    metrics:
    - type: f1
      value: 0.5167
      name: Macro-F1 (gold, 135-scope)
    - type: accuracy
      value: 0.5913
      name: Top-1 accuracy (gold, 135-scope)
---

# JobBERT-de (`jobbert-de-v2`)

A German job-title → **ESCO occupation** classifier: fine-tuned
[`deepset/gbert-base`](https://huggingface.co/deepset/gbert-base) with a closed
**135-way** sequence-classification head over the ESCO occupation scope of the
[Stellen-Atlas](https://huggingface.co/datasets/mischeiwiller/german-job-postings)
gold set. Given a German job title (and optional short text), it predicts the most
likely ESCO occupation URI.

Part of the **Stellen-Atlas** project — an open German/DACH jobs + skills corpus,
model, and demo.

## Intended use

- **Input:** a German job title (the production text is `title + derived description`;
  the corpus carries titles only, so titles dominate the signal).
- **Output:** one of 135 ESCO occupation URIs (`config.id2label`), argmax over the head.
- **Use cases:** occupation tagging / normalization of German vacancy titles, labour-market
  analytics, mapping postings onto the ESCO taxonomy (and via ESCO→ISCO onward to KldB).

### Out of scope

- **Occupation only.** It does **not** predict skills or the green-job flag — those are not
  title-checkable and were never gold-annotated (see the dataset card).
- **Closed 135-occupation scope.** Any occupation outside the gold scope is unreachable;
  the model will pick the nearest in-scope class instead. It is **not** an open-set tagger.
- Not a substitute for human review in high-stakes decisions (hiring, eligibility, pay).

## How to use

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

repo = "mischeiwiller/jobbert-de"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()

title = "Softwareentwickler (m/w/d) Backend"
inputs = tok(title, truncation=True, max_length=64, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
pred_id = int(logits.argmax(-1))
esco_uri = model.config.id2label[pred_id]   # an ESCO occupation URI
print(esco_uri)
```

## Method — why v2 beats v1

The honest two-step story behind this checkpoint:

- **v1 (weak-distillation, superseded):** a 2,146-way head trained on the corpus's own
  zero-shot E5 labels. A student distilled from a noisy teacher **tied but never beat**
  that teacher on macro-F1 (0.1899 vs the 0.1948 zero-shot baseline). Distilling weak
  labels caps the student at ~the teacher.
- **v2 (LLM-supervised, this checkpoint):** instead of class-balancing the same weak
  labels, the labels themselves were replaced. ~6,000 corpus postings (disjoint from the
  gold set) were each shown a scope-restricted German candidate menu (zero-shot top-10 ∩
  the 135 gold occupations) and labelled to the single best ESCO occupation — or "none" —
  by **Claude** (Anthropic), yielding **5,603 scope-valid silver labels** over 127 of the
  135 classes. `deepset/gbert-base` was then fine-tuned as a closed **135-way** classifier
  on this silver set.

The LLM-labeller was itself validated against the hand gold labels before the silver run:
**top-1 agreement 0.70** (vs 0.30 for the zero-shot teacher).

## Evaluation

Held-out gold set: `gold/gold.parquet` — 208 German titles, occupations annotated
**independently** of any model prediction (non-circular). Scored with a dependency-free
single-label macro-F1 / top-1 metric (cross-checked against scikit-learn). The zero-shot
baseline is `intfloat/multilingual-e5-base` **restricted to the same 135 occupations**, so
both predict over the identical closed label space (apples-to-apples).

| Metric | Zero-shot baseline (135-scope) | **`jobbert-de-v2`** | Delta |
|---|---|---|---|
| Macro-F1 (union of gold ∪ predicted) | 0.4832 | **0.5167** | **+0.0335** |
| Top-1 accuracy | 0.5240 | **0.5913** | **+0.0673** |
| Gold classes exactly right (F1 = 1.0) | 39 / 135 | 45 / 135 | +6 |
| Distinct occupations predicted | 99 | 98 | — |

`jobbert-de-v2` beats the fairly-scoped zero-shot on **both** metrics — the LLM-supervised
labels let the closed head learn the scope's occupations rather than echo the teacher's
mistakes.

> The original Phase-9.1 baseline (0.1948 macro-F1 / 0.3029 top-1) ranked zero-shot over
> the **full** ~3,000-occupation ESCO space. That is an unfair bar for a closed 135-way
> classifier, so the table above uses the re-baselined 135-scope numbers.

## Training

- **Base model:** `deepset/gbert-base` (German BERT).
- **Head:** 135-class sequence classification (ESCO occupation URIs in `config.id2label`).
- **Data:** Claude-labelled silver set (5,379 train / 224 val) over the 135-occupation scope.
- **Recipe:** 3 epochs, batch 64, max_len 64, lr 5e-5, class-balanced cross-entropy
  (inverse-frequency weights, clip 10.0) + label smoothing 0.1.
- **Validation:** macro-F1 0.7034, accuracy 0.7857 (in-distribution silver val).

## Limitations

- **127 / 135 scope classes have silver training rows** (8 had none), capping recall on
  those 8 occupations.
- **Title-only signal:** the corpus has no free-text descriptions, so terse or ambiguous
  titles are hard; predictions are strict exact-URI matches.
- **Single-source bias:** training and gold data derive from Bundesagentur für Arbeit
  vacancies — domain/register skews toward that source.
- **Silver labels are LLM-generated**, not human-verified at scale (gold-validated at 0.70
  agreement, not 1.0).
- ESCO `preferred_label` text in the corpus is mixed DE/EN; **only the URI is the join key.**

## Provenance & license

- **Model license:** Apache-2.0 (this fine-tune). Base model `deepset/gbert-base` is MIT.
- **Label taxonomy:** ESCO v1.2.0 (© European Union, CC-BY-4.0). Occupation URIs are ESCO
  `conceptUri` values.
- **Training corpus:** [`mischeiwiller/german-job-postings`](https://huggingface.co/datasets/mischeiwiller/german-job-postings)
  (derived from Bundesagentur für Arbeit Jobbörse API; see the dataset card for source ToS
  and the BA KldB non-commercial caveat).
- **Silver labels:** generated with Claude (Anthropic).

## Companion artifacts

Part of the **Stellen-Atlas** project (dataset + model + demo):

- **Dataset:** [`mischeiwiller/german-job-postings`](https://huggingface.co/datasets/mischeiwiller/german-job-postings)
  — the open German/DACH jobs + skills corpus this model was trained on.
- **Space:** [`mischeiwiller/stellen-atlas`](https://huggingface.co/spaces/mischeiwiller/stellen-atlas)
  — interactive demo: paste a German ad → ESCO occupation + skills + green share.

## Citation

```bibtex
@misc{stellen_atlas_jobbert_de,
  title  = {JobBERT-de: German job-title to ESCO occupation classifier},
  author = {Scheiwiller, Michael},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/mischeiwiller/jobbert-de}},
  note   = {Part of the Stellen-Atlas project; fine-tuned from deepset/gbert-base on Claude-supervised silver labels.}
}
```