---
language:
- hu
license: apache-2.0
base_model: SZTAKI-HLT/hubert-base-cc
pipeline_tag: text-classification
tags:
- hungarian
- corruption
- news-classification
datasets:
- K-Monitor/kmdb_classification
metrics:
- accuracy
- f1
---

# kmdb_classification_hubert_v5

Hungarian news article classifier for the
[K-Monitor press database](https://adatbazis.k-monitor.hu/): decides whether an
article belongs in the database, i.e. whether it describes a concrete case of
corruption, public procurement, public spending or transparency.

Finetuned from `SZTAKI-HLT/hubert-base-cc` on
[`K-Monitor/kmdb_classification`](https://huggingface.co/datasets/K-Monitor/kmdb_classification).

## Input format

The model is trained on one specific string. Anything else is out of
distribution:

```python
import re

def clean(text):
    return re.sub(r"\s+", " ", text or "").strip()

def format_article(title, description, url, text=""):
    domain = ".".join(url.split("/")[2].split(".")[-2:])
    # trim_title removes trailing newspaper names such as " - Telex"
    formatted = f"{trim_title(clean(title))}\n{clean(description)}\n({domain})"
    if text:
        formatted += "\n" + clean(text)[:800]
    return formatted
```

Tokenize with `max_length=320, truncation=True`. `label 1` means the article
belongs in the database; the probability of that class is the score. The
reference implementation lives in
[`sajtoadatbazis-automat`](https://github.com/k-monitor/sajtoadatbazis-automat)
(`webapp/auto_kmdb/processors/ClassificationProcessor.py` and
`train/article_classification/`).

## Results

Held-out test split of 9,914 articles, deduplicated by URL and by
title+description (40.2% positive):

| model | AUPR | accuracy @0.42 | precision | recall | F1 |
| --- | --- | --- | --- | --- | --- |
| `kmdb_classification_hubert_v2` | 0.8638 | 0.7119 | 0.590 | 0.926 | 0.7208 |
| **`kmdb_classification_hubert_v5`** | **0.9856** | **0.9502** | **0.939** | **0.937** | **0.9379** |

On the subset the previous version was never trained on (n=4,473), v2 scores
AUPR 0.7600 / accuracy 0.4581 and v5 scores AUPR 0.9683 / accuracy 0.9215.

The score is well calibrated around the 0.42 operating point used in
production; the precision/recall curve is flat between roughly 0.15 and 0.8.

| threshold | accuracy | precision | recall | F1 |
| --- | --- | --- | --- | --- |
| 0.15 | 0.9450 | 0.9096 | 0.9583 | 0.9333 |
| 0.374 | 0.9508 | 0.9367 | 0.9410 | 0.9388 |
| 0.42 | 0.9502 | 0.9386 | 0.9372 | 0.9379 |
| 0.60 | 0.9502 | 0.9488 | 0.9259 | 0.9372 |
| 0.80 | 0.9478 | 0.9611 | 0.9066 | 0.9330 |

## Training

Full finetune (not LoRA): 3 epochs, lr 2e-5, linear schedule, 300 warmup steps,
batch size 24, max length 320, weight decay 0.01, fp16, on a single RTX 3060.
The article body is dropped from 15% of training rows so the model does not
read a missing body as a label signal.

## Limitations

Positive examples come from the published K-Monitor database and negatives from
the autokmdb ingestion pipeline, so source and label are correlated in the
training data. Whitespace normalisation and title trimming remove the
mechanical traces of this (newline conventions, newspaper suffixes), and the
model's output moves by 0.0004 on average when fed unnormalised text — but the
correlation cannot be fully ruled out.

The companion category model
`K-Monitor/kmdb_classification_category_v5` is an SVM fitted on **this**
checkpoint's CLS embedding and is not interchangeable with the SVM for earlier
versions.