README.md

---
license: apache-2.0
base_model:
- Qwen/Qwen3-VL-8B-Instruct
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- chemistry
- ocsr
- image-to-smiles
- qwen3-vl
---

# Hiro-OCSR

Hiro-OCSR is an optical chemical structure recognition model that converts
chemical structure images into machine-readable SMILES.

This checkpoint is a full-parameter fine-tune of
[Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) for
image-to-SMILES generation. It is an early-training release intended for
evaluation and continued improvement, not a final or fully validated OCSR
model.

## Quickstart

Install the inference dependencies:

```bash
pip install "transformers>=4.57.1" accelerate torch torchvision pillow
```

The inference interface follows Qwen3-VL. Use the OCSR prompt exactly as shown
below:

```python
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "PatSnap/Hiro-OCSR"

model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "path/to/chemical-structure.png",
            },
            {
                "type": "text",
                "text": "Convert chemical structure in this image to SMILES.",
            },
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
inputs = inputs.to(model.device)

generated_ids = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=False,
)
generated_ids_trimmed = [
    output_ids[len(input_ids) :]
    for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)

print(output_text)
```

For the local SDK, chemical post-processing, structure validation, depiction,
and batch inference utilities, see the
[Hiro-OCSR source repository](https://github.com/hiro-ocsr/hiro-ocsr).

## Model details

- **Base model:** Qwen/Qwen3-VL-8B-Instruct
- **Fine-tuning method:** full-parameter fine-tuning
- **Task:** optical chemical structure recognition and image-to-SMILES generation
- **Architecture:** Qwen3VLForConditionalGeneration
- **Recommended prompt:** `Convert chemical structure in this image to SMILES.`
- **Release stage:** early training

## Limitations

This model may produce inaccurate, incomplete, misrecognized, improperly
canonicalized, or chemically invalid SMILES. Errors may include missing atoms
or bonds, incorrect stereochemistry, charges, isotopes, salts, abbreviations,
R-groups, variable attachments, repeat units, or reaction components.

Model outputs must be reviewed and validated before use in chemical analysis,
patent work, regulatory submissions, laboratory workflows, safety-critical
applications, database curation, or other professional contexts. Users are
responsible for ensuring they have the necessary rights to process input images
and documents.

See the full [Disclaimer](DISCLAIMER.md) before using the model.

## License and attribution

This model repository is made available under the Apache License 2.0. The model
is based on
[Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct);
review the base model card and its applicable terms as well.

See [NOTICE](NOTICE) for copyright, trademark, and attribution information.

## Related resources

- [Hiro-OCSR source code](https://github.com/hiro-ocsr/hiro-ocsr)
- [Hiro-OCSR Real 24K dataset](https://huggingface.co/datasets/PatSnap/hiro-ocsr-real-24k)
- [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct)