README.md

---
license: other
library_name: transformers
tags:
- chemistry
- molecular-structure
- reaction-diagram
- robustrdp
base_model: Qwen/Qwen2.5-VL-3B-Instruct
pipeline_tag: image-text-to-text
---

# RobustRDP

Fine-tuned model for the paper: *RobustRDP: Advancing Reaction Diagram Parsing via Synthetic-to-Real Data Scaling and Robustness-Oriented Training*.

## Description

This model is a fine-tuned checkpoint based on **Qwen2.5-VL-3B-Instruct**, trained with a three-stage pipeline:

1. **Pretrain stage**: Synthetic data pretraining on large-scale synthetic reaction diagrams.
2. **SFT stage**: Supervised fine-tuning with three specialized tasks:
   - **Vanilla Reaction Parsing (VRP)**: Standard reaction diagram parsing
   - **Region-Guided Reaction Parsing (RGRP)**: Region-aware parsing with spatial guidance
   - **Prefix-Perturbed Reaction Parsing (PPRP)**: Robustness-oriented parsing with prefix perturbations
3. **DPO stage**: Direct Preference Optimization to further align model outputs with ground-truth annotations.

## Training Details

| Config | Pretrain | SFT | DPO |
|--------|----------|-----|-----|
| Base Model | Qwen2.5-VL-3B-Instruct | Qwen2.5-VL-3B-Instruct | Qwen2.5-VL-3B-Instruct |
| Learning Rate | 1.0×10⁻⁶ | 1.0×10⁻⁵ | 3.0×10⁻⁷ |
| Batch Size | 16 | 4 | 64 |
| Epochs | 1 | 1 | 1 |
| Scheduler | Cosine (warmup 0.03) | Cosine (warmup 0.03) | Cosine (warmup 0.03) |
| Optimizer | AdamW | AdamW | AdamW |
| Trainable Params | LLM only | Full (vision + LLM) | LLM only |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Jingcz/RobustRDP"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
```

## Related Resources

- **Dataset**: [RxnLabelData](https://huggingface.co/datasets/Jingcz/RxnLabelData)
- **Annotation Platform**: [RxnLabel](https://github.com/jaydetang/RxnLabel)