license: apache-2.0
base_model:
- Qwen/Qwen3-VL-8B-Instruct
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- chemistry
- ocsr
- image-to-smiles
- qwen3-vl
Hiro-OCSR
Hiro-OCSR is an optical chemical structure recognition model that converts chemical structure images into machine-readable SMILES.
This checkpoint is a full-parameter fine-tune of Qwen3-VL-8B-Instruct for image-to-SMILES generation. It is an early-training release intended for evaluation and continued improvement, not a final or fully validated OCSR model.
Quickstart
Install the inference dependencies:
pip install "transformers>=4.57.1" accelerate torch torchvision pillow
The inference interface follows Qwen3-VL. Use the OCSR prompt exactly as shown below:
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
model_id = "PatSnap/Hiro-OCSR"
model = Qwen3VLForConditionalGeneration.from_pretrained(
model_id,
dtype="auto",
device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)
messages = [
{
"role": "user",
"content": [
{
"type": "image",
"image": "path/to/chemical-structure.png",
},
{
"type": "text",
"text": "Convert chemical structure in this image to SMILES.",
},
],
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
)
inputs = inputs.to(model.device)
generated_ids = model.generate(
**inputs,
max_new_tokens=1024,
do_sample=False,
)
generated_ids_trimmed = [
output_ids[len(input_ids) :]
for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)
print(output_text)
For the local SDK, chemical post-processing, structure validation, depiction, and batch inference utilities, see the Hiro-OCSR source repository.
Model details
- Base model: Qwen/Qwen3-VL-8B-Instruct
- Fine-tuning method: full-parameter fine-tuning
- Task: optical chemical structure recognition and image-to-SMILES generation
- Architecture: Qwen3VLForConditionalGeneration
- Recommended prompt:
Convert chemical structure in this image to SMILES. - Release stage: early training
Limitations
This model may produce inaccurate, incomplete, misrecognized, improperly canonicalized, or chemically invalid SMILES. Errors may include missing atoms or bonds, incorrect stereochemistry, charges, isotopes, salts, abbreviations, R-groups, variable attachments, repeat units, or reaction components.
Model outputs must be reviewed and validated before use in chemical analysis, patent work, regulatory submissions, laboratory workflows, safety-critical applications, database curation, or other professional contexts. Users are responsible for ensuring they have the necessary rights to process input images and documents.
See the full Disclaimer before using the model.
License and attribution
This model repository is made available under the Apache License 2.0. The model is based on Qwen3-VL-8B-Instruct; review the base model card and its applicable terms as well.
See NOTICE for copyright, trademark, and attribution information.