Image-to-Text
Transformers
Safetensors
molparser_vision_encoder_decoder
image-text-to-text
chemistry
custom_code
AI4Industry commited on
Commit
4509e64
Β·
verified Β·
1 Parent(s): 29d8d1e

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +102 -0
README.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ pipeline_tag: image-to-text
4
+ tags:
5
+ - chemistry
6
+ - image-to-text
7
+ datasets:
8
+ - UniParser/MolParser-7M
9
+ - UniParser/MolGallery
10
+ ---
11
+
12
+ # MolParser Mobile
13
+
14
+ <p align="center">
15
+ πŸ’» <a href="https://github.com/dptech-corp/MolParser">Github</a> |
16
+ πŸ“„ <a>Report (Coming soon...)</a> |
17
+ πŸš€ <a href="https://ocsr.dp.tech/">Demo</a>
18
+ </p>
19
+
20
+ **MolParser-Mobile** is the next-generation lightweight version of **MolParser**, designed for **ultrafast Optical Chemical Structure Recognition (OCSR)**. It converts molecular structure images into **E-SMILES** representations with significantly higher throughput while maintaining state-of-the-art recognition accuracy.
21
+
22
+ Compared with MolParser 1.0, MolParser-Mobile reduces the model size by **95.4%** (216M β†’ **9.98M** parameters) and increases inference throughput by **38Γ—**, reaching **1,520 molecules/second** on a single NVIDIA RTX 4090D GPU. Despite its compact architecture, it achieves better accuracy on multiple real-world OCSR benchmarks, making it suitable for large-scale chemical literature mining and industrial deployment.
23
+
24
+
25
+ ## πŸš€ Highlights
26
+
27
+ * **38Γ— faster** than MolParser 1.0 (**1,520 Mol/s** on RTX 4090D)
28
+ * **9.98M parameters** (**95.4% smaller** than MolParser 1.0)
29
+ * **State-of-the-art** accuracy on challenging real-world OCSR benchmarks (BioVista)
30
+ * Designed for **web-scale** chemical literature mining
31
+ * Converts molecular images directly into **E-SMILES** strings
32
+
33
+
34
+ ## πŸ“Š Performance
35
+
36
+ | Model | Parameters | Throughput (RTX 4090D) | Uni-Parser Bench | BioVista | WildMol-10k |
37
+ | -------------------- | -----------------: | -------------------------: | -----------------: | -----------------: | -------------: |
38
+ | Gemma4-31B | 31B | OOM | 0.073 | 0.118 | 0.232 |
39
+ | Qwen3.5-397B-A17B | 397B | OOM | 0.272 | 0.347 | 0.303 |
40
+ | MolParser 1.0 | 216M | 39.8 Mol/s | 0.800 | 0.703 | **0.769** |
41
+ | **MolParser-Mobile** | **9.98M (-95.4%)** | **1,520 Mol/s (+3819.1%)** | **0.823 (+0.023)** | **0.801 (+0.098)** | 0.734 (-0.035) |
42
+
43
+
44
+ ## ⚑ Usage
45
+
46
+ ### Option 1. πŸ€— Transformers
47
+ Load MolParser-Mobile directly with the Hugging Face transformers library.
48
+
49
+ ```python
50
+ import torch
51
+ from PIL import Image
52
+ from transformers import AutoModelForImageTextToText, AutoProcessor
53
+
54
+ repo_id = "UniParser/MolParser-Mobile"
55
+ device = "cuda" if torch.cuda.is_available() else "cpu"
56
+ dtype = torch.float16 if device == "cuda" else torch.float32
57
+
58
+ processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)
59
+ model = AutoModelForImageTextToText.from_pretrained(
60
+ repo_id,
61
+ dtype=dtype,
62
+ trust_remote_code=True,
63
+ ).to(device).eval()
64
+
65
+ image = Image.open("mol.png").convert("RGB")
66
+ inputs = processor(images=image, return_tensors="pt")
67
+ inputs = {k: v.to(device, dtype=dtype) for k, v in inputs.items()}
68
+
69
+ output_ids = model.generate(**inputs, max_length=256, num_beams=1, do_sample=False)
70
+ caption = processor.batch_decode(output_ids, skip_special_tokens=True)[0]
71
+ print(caption)
72
+ ```
73
+
74
+ ### Option 2. MolParser Library (Recommended)
75
+
76
+ The [MolParser library](https://github.com/dptech-corp/MolParser) provides a convenient interface for molecule detection and recognition.
77
+
78
+ Clone the repository and install the package:
79
+
80
+ ```bash
81
+ git clone https://github.com/dptech-corp/MolParser.git
82
+ cd MolParser
83
+ pip install -e .
84
+ ```
85
+
86
+ Then run:
87
+
88
+ ```python
89
+ from molparser import MolParser
90
+
91
+ parser = MolParser()
92
+
93
+ result = parser.parse(
94
+ "mol.png",
95
+ rec_only=True, # Recognition only
96
+ )
97
+ ```
98
+
99
+
100
+ ## πŸ“– Citation
101
+ If you use this model, please cite:
102
+