Instructions to use numind/NuExtract-2.0-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use numind/NuExtract-2.0-2B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="numind/NuExtract-2.0-2B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("numind/NuExtract-2.0-2B") model = AutoModelForImageTextToText.from_pretrained("numind/NuExtract-2.0-2B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use numind/NuExtract-2.0-2B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "numind/NuExtract-2.0-2B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "numind/NuExtract-2.0-2B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/numind/NuExtract-2.0-2B
- SGLang
How to use numind/NuExtract-2.0-2B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "numind/NuExtract-2.0-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "numind/NuExtract-2.0-2B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "numind/NuExtract-2.0-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "numind/NuExtract-2.0-2B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use numind/NuExtract-2.0-2B with Docker Model Runner:
docker model run hf.co/numind/NuExtract-2.0-2B
A thank you and a question
First, thank you! This is literally exactly what i needed for a problem im having. Like, eerie how perfectly the release was timed for my use-case!
The vision aspect is just... awesome.
One question i have, however, is there a way to specify arrays of objects? Right now, ive been doing something along the lines of:
{
"title": ["string"],
"first_name": ["string"]
}
But, i frequently (majority of runs with each size) run into the issue of the model giving an uneven number of results for each (e.g. 3 title but 2 in first_name).
Im currently using one example for ICL.
Is there a way to structure it s.t. I am making it clear they go together? Or is this out of scope.
Also fun fact im like 15 minutes away from cambridge rn haha
Glad to hear the model is useful for you!
Yes, you can define arrays and nested objects in the template, for example:
{
"people": [
{
"title": ["string"],
"first_name": ["string"]
}
]
}
Thanks so much!! That is so convenient that it just works.
Out of curiosity, is there a way to indicate optional fields?
Technically no, but we trained the models to encourage putting null or an empty array [] for a particular field if it can't find the info or the field is not relevant to the document.
Oh that makes sense. So i can just put everything and have IT figure out what is optional!
This might not fully match the main discussion, and I feel a bit shy to post it there, so I'm sharing it here as a short sub-discussion :)
I haven’t evaluated the performance yet, but I’ve developed a model that automatically generates a JSON template from plain text.
Unlike typical text-to-json models that require predefined templates, this one builds the template directly from the input text.
I’d love for you to give it a try!