CauSight: Learning to Supersense for Visual Causal Discovery

This repository contains the CauSight model, a novel vision-language model designed to perform visual causal discovery through causally aware reasoning. CauSight enables AI systems to infer cause-and-effect relations among visual entities across diverse scenarios, moving beyond mere perception. It integrates training data curation, Tree-of-Causal-Thought (ToCT) for synthesizing reasoning trajectories, and reinforcement learning with a designed causal reward. Experiments demonstrate that CauSight significantly outperforms models like GPT-4.1 on visual causal discovery.

This work is introduced in the following paper:

CauSight: Learning to Supersense for Visual Causal Discovery [📄 arXiv]

Project Page and Code: https://github.com/OpenCausaLab/CauSight

🔧 User Guide

1. Clone the Repository

git clone https://github.com/OpenCausaLab/CauSight.git
cd CauSight

2. Set Up the Environment

We recommend using conda:

conda create -n causight python=3.10
conda activate causight

pip install -r requirements.txt
pip install -e .

3. Download the Dataset (VCG-32K)

mkdir -p VCG-32K
pip install huggingface_hub

hf login
hf download OpenCausaLab/VCG-32K \
    --repo-type dataset \
    --local-dir ./VCG-32K

tar -xzf ./VCG-32K/COCO/images.tar.gz -C ./VCG-32K/COCO
tar -xzf ./VCG-32K/365/images.tar.gz -C ./VCG-32K/365

4. Download the CauSight Model

mkdir -p model
huggingface-cli download OpenCausaLab/CauSight \
    --repo-type model \
    --local-dir ./model

5. Evaluation

Start the model server, then run inference:

bash model_server.sh
python run_inference.py

6. Tree-of-Causal-Thought (If you want to make your own SFT data with ToCT.)

bash model_server.sh
python run.py

Citation

If you find our work helpful or inspiring, please consider citing it:

@article{zhang2025causight,
  title={CauSight: Learning to Supersense for Visual Causal Discovery},
  author={Zhang, Yize and Chen, Meiqi and Chen, Sirui and Peng, Bo and Zhang, Yanxi and Li, Tianyu and Lu, Chaochao},
  journal={arXiv preprint arXiv:2512.01827},
  year={2025},
  url={https://arxiv.org/abs/2512.01827}
}