Instructions to use docling-project/SmolDocling-256M-preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use docling-project/SmolDocling-256M-preview with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="docling-project/SmolDocling-256M-preview") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("docling-project/SmolDocling-256M-preview") model = AutoModelForMultimodalLM.from_pretrained("docling-project/SmolDocling-256M-preview") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use docling-project/SmolDocling-256M-preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "docling-project/SmolDocling-256M-preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docling-project/SmolDocling-256M-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/docling-project/SmolDocling-256M-preview
- SGLang
How to use docling-project/SmolDocling-256M-preview with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "docling-project/SmolDocling-256M-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docling-project/SmolDocling-256M-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "docling-project/SmolDocling-256M-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docling-project/SmolDocling-256M-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use docling-project/SmolDocling-256M-preview with Docker Model Runner:
docker model run hf.co/docling-project/SmolDocling-256M-preview
Got an error of AttributeError: 'str' object has no attribute 'with_suffix'
I got an error of
File "/home/ubuntu/workspace/docling/.venv/lib/python3.12/site-packages/docling_core/types/doc/document.py", line 2747, in _get_output_paths
artifacts_dir = filename.with_suffix("")
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'with_suffix'
Any help?
Thank you.
Hello Elias,
Can you please post what code or line that triggered this?
I got the same error, in this line:
doc.save_as_markdown(output_path_md)
My solution:
output_path_md = os.path.join(OUTPUT_DIR, output_filename_md)
markdown_content = doc.export_to_markdown()
with open(output_path_md, "w") as file:
file.write(markdown_content)
For me, the following worked:
from pathlib import Path
results = doc.save_as_html(Path(output_filename))
Hello Elias,
Can you please post what code or line that triggered this?
When I run the example code on Model Card page, Fast Batch Inference Using VLLM, I got the error following in the docling-core module.
Traceback (most recent call last):
File "/home/ubuntu/workspace/docling/test.py", line 61, in
doc.save_as_markdown(output_path_md)
File "/home/ubuntu/workspace/docling/.venv/lib/python3.12/site-packages/docling_core/types/doc/document.py", line 2575, in save_as_markdown
artifacts_dir, reference_path = self._get_output_paths(filename, artifacts_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/workspace/docling/.venv/lib/python3.12/site-packages/docling_core/types/doc/document.py", line 2749, in _get_output_paths
artifacts_dir = artifacts_dir.with_name(artifacts_dir.name + "_artifacts")
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'with_name'
Sorry for the confusion, the code examples in the readme have been fixed!