README.md

---
license: llama2
base_model:
- lmsys/vicuna-7b-v1.5
tags:
- vision-language-model
- TEM,microscopy
- materials-science
- llava
- scientific-VLM
language:
- en
pipeline_tag: image-text-to-text
---


# ATOMIC-LLaVA

ATOMIC-LLaVA is a domain-specific Vision-Language Model for Transmission Electron Microscopy (TEM), fine-tuned from LLaVA-v1.5-7B (Vicuna-v1.5-7B) using a two-stage training pipeline on 32,564 TEM subfigures collected from Nature portfolio journals.

This model is introduced in the ECCV 2026 paper:

> **ATOMIC: A Domain-Specific Vision-Language Model for Transmission Electron Microscopy**

For code, evaluation scripts, and dataset, please refer to our GitHub repository:
👉 [https://github.com/SemiMRTLab-NCKU/ATOMIC](https://github.com/SemiMRTLab-NCKU/ATOMIC)

---

## Model Details

| | |
|---|---|
| **Base Model** | LLaVA-v1.5-7B (Vicuna-v1.5-7B) |
| **Training Stage** | Stage 1 (alignment) + Stage 2 (instruction tuning) |
| **Training Data** | 120K Stage 1 pairs + 60K Stage 2 conversations |
| **Domain** | Transmission Electron Microscopy (TEM) |
| **Modalities** | CTEM, HR-TEM, STEM, Diffraction |

---

## Important: Inference Requirements

ATOMIC-LLaVA is built on LLaVA and **cannot be loaded directly via `transformers`**. Inference requires the LLaVA repository.

**Step 1 — Clone LLaVA:**
```bash
git clone https://github.com/haotian-liu/LLaVA.git
cd LLaVA
pip install -e .
```

**Step 2 — Download weights:**
```python
from huggingface_hub import snapshot_download
snapshot_download(repo_id="LabSmart/ATOMIC-LLaVA", local_dir="./ATOMIC-LLaVA")
```

**Step 3 — Run inference using our evaluation scripts:**

Please refer to `evaluation/` in our GitHub repository for inference and evaluation scripts.

---

## Training Data

Training data is available on HuggingFace:
👉 [https://huggingface.co/datasets/LabSmart/ATOMIC_dataset](https://huggingface.co/datasets/LabSmart/ATOMIC_dataset)

---

## Citation

```bibtex
@inproceedings{atomic2026eccv,
  title     = {ATOMIC: A Domain-Specific Vision-Language Model
               for Transmission Electron Microscopy},
  author    = {Tu, C. and Hsu, Shu-han and others},
  booktitle = {Proceedings of ECCV 2026},
  year      = {2026},
  note      = {BibTeX will be updated upon publication}
}
```

---

## License

This model is released under the [LLaMA 2 Community License](https://ai.meta.com/llama/license/). It is intended for academic research purposes only and may not be used for commercial purposes.