Hugging Face's logo

starride-teklia
/
DAI-NER-nested

Image-to-Text
PEFT
Safetensors
French
qwen3_5
QWEN
ner
historical
handwritten
Model card Files
xet
 Community
DAI-NER-nested
File size: 3,903 Bytes
efde046
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
---
base_model: QWEN/Qwen3.5-9B
library_name: peft
tags:
- QWEN
- ner
- historical
- handwritten
language:
- fr
pipeline_tag: image-to-text
---

# QWEN3.5 - DAI NER Nested Model

## Introduction

This version of Qwen3.5-9B is specialized for NER on French parish records from the 16th-18th centuries.
Ref: https://redmine.teklia.com/issues/13357

## Training

The model is a [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) fine-tuned for NER on French parish records using LoRA.

Parameters:
- Image width: 1500 pixels
- LoRa rank: 8
- LoRa alpha: 32
- Epochs: 10 (about 4k steps)

## Usage

Here we show a code snippet to show you how to use the model with `transformers` and `qwen_vl_utils`:

* Prediction script

```python
from transformers import AutoProcessor, AutoModelForMultimodalLM
from qwen_vl_utils import process_vision_info
from pathlib import Path
import torch

# Load QWEN
model = AutoModelForMultimodalLM.from_pretrained(
    "starride-teklia/DAI-NER-nested",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("starride-teklia/DAI-NER-nested")

# Prompt
SYSTEM = Path("system.txt").read_text()		# Available in the model directory
IMAGE = "record.jpg"		                     # https://europe.iiif.teklia.com/iiif/2/geneanet%2FArdennes_BMS%2F379692%2F00100.jpg/1425,793,1252,284/full/0/default.jpg
HTR_TRANSCRIPTION = "Mort de Pierre Soyer gardien de la redoutte du prez d'an...gne et ligne de cette frontiere\n\nL'an mil sept cens et neuf le vingt neuvième jour du mois de jeanvier a esté decedée en cette paroisse Pierre Soyer gardien au pre d'A[...]gne de la paroisse d'Aumont agee de quarante deux ans ou environs lequel a esté inhumé le trantième dudit mois dans le cimetier de cette paroisse ou nous lavons conduit avec les ceremonies accoustumée en présence avec les témoins qui ont signé avec nous\n\nJames Gileux T Stenva prb Mathieu Lallement."

messages = [
    {
        "role": "system",
        "content": [
           {
               "type": "text",
               "text": SYSTEM
           }
        ]
    },
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "record.jpg"
            },
            {
                "type": "text",
                "text": f"Convert this record in JSON. To help you, here is the transcription produced by an OCR model: {HTR_TRANSCRIPTION}"
            },
        ],
    }
]

# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to("cuda")

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=1024)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]
print(output_text)

```

* Output

```
{"événements": {"décès": {"année": "mil sept cens et neuf", "jour": "vingt neuvième", "mois": "janvier", "lieu": "en cette paroisse"}, "acte": {"jour": "trantième", "mois": "dudit mois", "lieu": "cette paroisse"}}, "individus": [{"rôle": "défunt", "attributes": {"prénom": "Pierre", "nom": "Soyer", "profession": "gardien au pre d'A[...]gne", "âge": "quarante deux ans ou environs"}}, {"rôle": "temoin_1", "attributes": {"prénom": "James", "nom": "Gileux"}}, {"rôle": "temoin_2", "attributes": {"prénom": "T", "nom": "Stenva"}}, {"rôle": "temoin_3", "attributes": {"prénom": "Mathieu", "nom": "Lallement"}}]}
```