|
|
--- |
|
|
library_name: peft |
|
|
base_model: LSX-UniWue/ModernGBERT_1B |
|
|
tags: |
|
|
- base_model:adapter:LSX-UniWue/ModernGBERT_1B |
|
|
- lora |
|
|
- transformers |
|
|
- token-classification |
|
|
--- |
|
|
|
|
|
|
|
|
# ModernGBERT Redewiedergabe Tagger |
|
|
|
|
|
This model is a token classifier that recognizes German speech, thought and writing representation (STWR), that is being used in [LLpro](https://github.com/cophi-wue/LLpro). Besides the *medium* (speech, thought, writing) the model also predicts the *type* (direct, free indirect, indirect, reported) by providing 36 classification outputs (3 media × 4 types × B,I,O). |
|
|
|
|
|
| STWR type | Example | Translation | |
|
|
|--------------------------------|-------------------------------------------------------------------------|----------------------------------------------------------| |
|
|
| direct | Dann schrieb er: **"Ich habe Hunger."** | Then he wrote: **"I'm hungry."** | |
|
|
| free indirect ('erlebte Rede') | Er war ratlos. **Woher sollte er denn hier bloß ein Mittagessen bekommen?** | He was at a loss. **Where should he ever find lunch here?** | |
|
|
| indirect | Sie fragte, **wo das Essen sei.** | She asked **where the food was.** | |
|
|
| reported | **Sie dachte über das Mittagessen.** | **She thought about lunch.** | |
|
|
|
|
|
This model is a fine-tuned version of [LSX-UniWue/ModernGBERT_1B](https://huggingface.co/LSX-UniWue/ModernGBERT_1B) on the [REDEWIEDERGABE corpus](https://github.com/redewiedergabe/corpus) ([Annotation guidelines](http://redewiedergabe.de/richtlinien/richtlinien.html)). |
|
|
|
|
|
[Training Script](https://github.com/cophi-wue/LLpro/blob/main/contrib/train_redewiedergabe.py). |
|
|
|
|
|
### Performance |
|
|
|
|
|
We report simplified F1 scores on a binarized variant (O vs B/I) for each speech type resp. medium. |
|
|
|
|
|
| Type, Medium | F1 Score | support | |
|
|
|:----------------------|---------:|---------:| |
|
|
| direct.speech | 0.96 | 13598 | |
|
|
| direct.thought | 0.79 | 715 | |
|
|
| direct.writing | 0.19 | 996 | |
|
|
| indirect.speech | 0.77 | 1226 | |
|
|
| indirect.thought | 0.71 | 802 | |
|
|
| indirect.writing | 0.00 | 11 | |
|
|
| freeIndirect.speech | 0.73 | 198 | |
|
|
| freeIndirect.thought | 0.45 | 251 | |
|
|
| freeIndirect.writing | – | – | |
|
|
| reported.speech | 0.69 | 1684 | |
|
|
| reported.thought | 0.59 | 799 | |
|
|
| reported.writing | 0.56 | 135 | |
|
|
| *micro avg* | *0.86* | 20415 | |
|
|
| *macro avg* | *0.54* | 20415 | |
|
|
|
|
|
### Demo Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForTokenClassification, AutoTokenizer |
|
|
|
|
|
speech_labels = [ |
|
|
"direct.speech", |
|
|
"direct.thought", |
|
|
"direct.writing", |
|
|
"indirect.speech", |
|
|
"indirect.thought", |
|
|
"indirect.writing", |
|
|
"freeIndirect.speech", |
|
|
"freeIndirect.thought", |
|
|
"freeIndirect.writing", |
|
|
"reported.speech", |
|
|
"reported.thought", |
|
|
"reported.writing", |
|
|
] |
|
|
|
|
|
text = """ |
|
|
Dann schrieb er: 'Ich habe Hunger." |
|
|
Er war ratlos. Woher sollte er denn hier bloß ein Mittagessen bekommen? |
|
|
Sie fragte, wo das Essen sei. |
|
|
Sie dachte über das Mittagessen.""" |
|
|
|
|
|
model_id = 'aehrm/moderngbert-redewiedergabe' |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
|
model = AutoModelForTokenClassification.from_pretrained(model_id, num_labels=3*len(speech_labels)) |
|
|
|
|
|
inputs = tokenizer(text, return_tensors='pt') |
|
|
out = model(**inputs) |
|
|
|
|
|
batch_size, seq_len, _ = out.logits.shape |
|
|
prediction = out.logits.reshape(batch_size, seq_len, 12, 3).argmax(-1) |
|
|
|
|
|
for i, speech_label in enumerate(speech_labels): |
|
|
for tok, pred in zip(tokenizer.convert_ids_to_tokens(inputs['input_ids'][0]), prediction[0,:,i]): |
|
|
pred = 'OBI'[pred] |
|
|
print(tok, pred, speech_label if pred != 'O' else '') |
|
|
``` |
|
|
|
|
|
### Training results |
|
|
|
|
|
F1 Score refers to the micro average over all 36 classification outputs. |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | F1 Score | |
|
|
|:-------------:|:-----:|:----:|:---------------:|:--------:| |
|
|
| 0.1951 | 1.0 | 193 | 0.1332 | 0.1180 | |
|
|
| 0.0885 | 2.0 | 386 | 0.2474 | 0.2724 | |
|
|
| 0.0417 | 3.0 | 579 | 0.1455 | 0.4604 | |
|
|
| 0.0753 | 4.0 | 772 | 0.1399 | 0.5522 | |
|
|
| 0.0277 | 5.0 | 965 | 0.1447 | 0.6170 | |
|
|
| 0.0238 | 6.0 | 1158 | 0.1770 | 0.6200 | |
|
|
| 0.0153 | 7.0 | 1351 | 0.2257 | 0.6930 | |
|
|
| 0.009 | 8.0 | 1544 | 0.6031 | 0.7336 | |
|
|
| 0.0108 | 9.0 | 1737 | 0.4965 | 0.7428 | |
|
|
| 0.0066 | 10.0 | 1930 | 0.4575 | 0.7492 | |
|
|
| 0.0058 | 11.0 | 2123 | 0.7781 | 0.7983 | |
|
|
| 0.006 | 12.0 | 2316 | 0.8648 | 0.8062 | |
|
|
| 0.0043 | 13.0 | 2509 | 1.0377 | 0.8148 | |
|
|
| 0.0033 | 14.0 | 2702 | 1.3040 | 0.8217 | |
|
|
| 0.0025 | 15.0 | 2895 | 1.2637 | 0.8359 | |
|
|
| 0.003 | 16.0 | 3088 | 1.3230 | 0.8477 | |
|
|
| 0.0019 | 17.0 | 3281 | 1.9811 | 0.8439 | |
|
|
| 0.0014 | 18.0 | 3474 | 2.1191 | 0.8482 | |
|
|
| 0.0011 | 19.0 | 3667 | 2.3599 | 0.8510 | |
|
|
| 0.0009 | 20.0 | 3860 | 2.4453 | 0.8528 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- PEFT 0.17.0 |
|
|
- Transformers 4.55.2 |
|
|
- Pytorch 2.8.0+cu128 |
|
|
- Datasets 2.21.0 |
|
|
- Tokenizers 0.21.4 |
|
|
|
