Image-to-Text
Transformers
Safetensors
qwen3_5
image-text-to-text
vision-language
vlm
document-understanding
structured-extraction
information-extraction
ocr
document-to-markdown
markdown
rag
reasoning
multilingual
conversational
Instructions to use numind/NuExtract3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use numind/NuExtract3 with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="numind/NuExtract3") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("numind/NuExtract3") model = AutoModelForImageTextToText.from_pretrained("numind/NuExtract3") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -114,7 +114,7 @@ Structured extraction takes as inputs:
|
|
| 114 |
|
| 115 |
1. An input document, which can be text, image, or both;
|
| 116 |
2. A JSON template describing the information to extract;
|
| 117 |
-
3. (Optional) Instructions, allowing to specify expected output formats or values;
|
| 118 |
4. (Optional) In-Context Learning (ICL) examples.
|
| 119 |
|
| 120 |
### Input JSON template
|
|
@@ -356,7 +356,8 @@ response = client.chat.completions.create(
|
|
| 356 |
],
|
| 357 |
extra_body={
|
| 358 |
"chat_template_kwargs": {
|
| 359 |
-
"template": json.dumps(template
|
|
|
|
| 360 |
"enable_thinking": False
|
| 361 |
}
|
| 362 |
}
|
|
|
|
| 114 |
|
| 115 |
1. An input document, which can be text, image, or both;
|
| 116 |
2. A JSON template describing the information to extract;
|
| 117 |
+
3. (Optional) Instructions, allowing to specify expected output formats or values, to provide with the `instructions` chat template kwarg;
|
| 118 |
4. (Optional) In-Context Learning (ICL) examples.
|
| 119 |
|
| 120 |
### Input JSON template
|
|
|
|
| 356 |
],
|
| 357 |
extra_body={
|
| 358 |
"chat_template_kwargs": {
|
| 359 |
+
"template": json.dumps(template),
|
| 360 |
+
"instructions": "Specify the time for the `date` entry only if it is present, otherwise only output the date component.",
|
| 361 |
"enable_thinking": False
|
| 362 |
}
|
| 363 |
}
|