README.md

---
license: apache-2.0
language:
  - en
tags:
  - scene-graph-generation
  - object-detection
  - visual-relationship-detection
  - pytorch
  - yolo
pipeline_tag: object-detection
library_name: sgg-benchmark
model-index:
  - name: REACT++ yolo12l
    results:
      - task:
          type: object-detection
          name: Scene Graph Detection
        dataset:
          name: PSG
          type: psg
        metrics:
          - type: mR@20
            value: 23.2
            name: mR@20
          - type: R@20
            value: 30.99
            name: R@20
          - type: F1@20
            value: 26.53
            name: F1@20
          - type: mR@50
            value: 25.49
            name: mR@50
          - type: R@50
            value: 35.3
            name: R@50
          - type: F1@50
            value: 29.6
            name: F1@50
          - type: mR@100
            value: 26.45
            name: mR@100
          - type: R@100
            value: 36.68
            name: R@100
          - type: F1@100
            value: 30.74
            name: F1@100
          - type: e2e_latency_ms
            value: 19.6
            name: e2e_latency_ms
  - name: REACT++ yolo12m
    results:
      - task:
          type: object-detection
          name: Scene Graph Detection
        dataset:
          name: PSG
          type: psg
        metrics:
          - type: mR@20
            value: 22.74
            name: mR@20
          - type: R@20
            value: 32.69
            name: R@20
          - type: F1@20
            value: 26.82
            name: F1@20
          - type: mR@50
            value: 25.21
            name: mR@50
          - type: R@50
            value: 37.2
            name: R@50
          - type: F1@50
            value: 30.05
            name: F1@50
          - type: mR@100
            value: 26.08
            name: mR@100
          - type: R@100
            value: 38.58
            name: R@100
          - type: F1@100
            value: 31.12
            name: F1@100
          - type: e2e_latency_ms
            value: 15.7
            name: e2e_latency_ms
  - name: REACT++ yolo12s
    results:
      - task:
          type: object-detection
          name: Scene Graph Detection
        dataset:
          name: PSG
          type: psg
        metrics:
          - type: mR@20
            value: 21.12
            name: mR@20
          - type: R@20
            value: 29.28
            name: R@20
          - type: F1@20
            value: 24.54
            name: F1@20
          - type: mR@50
            value: 23.21
            name: mR@50
          - type: R@50
            value: 33.48
            name: R@50
          - type: F1@50
            value: 27.41
            name: F1@50
          - type: mR@100
            value: 23.77
            name: mR@100
          - type: R@100
            value: 34.74
            name: R@100
          - type: F1@100
            value: 28.23
            name: F1@100
          - type: e2e_latency_ms
            value: 12.2
            name: e2e_latency_ms
  - name: REACT++ yolo12n
    results:
      - task:
          type: object-detection
          name: Scene Graph Detection
        dataset:
          name: PSG
          type: psg
        metrics:
          - type: mR@20
            value: 16.88
            name: mR@20
          - type: R@20
            value: 26.88
            name: R@20
          - type: F1@20
            value: 20.74
            name: F1@20
          - type: mR@50
            value: 18.65
            name: mR@50
          - type: R@50
            value: 30.61
            name: R@50
          - type: F1@50
            value: 23.17
            name: F1@50
          - type: mR@100
            value: 19.5
            name: mR@100
          - type: R@100
            value: 31.8
            name: R@100
          - type: F1@100
            value: 24.17
            name: F1@100
          - type: e2e_latency_ms
            value: 11.4
            name: e2e_latency_ms
  - name: REACT++ yolov8m
    results:
      - task:
          type: object-detection
          name: Scene Graph Detection
        dataset:
          name: PSG
          type: psg
        metrics:
          - type: mR@20
            value: 22.75
            name: mR@20
          - type: R@20
            value: 30.69
            name: R@20
          - type: F1@20
            value: 26.13
            name: F1@20
          - type: mR@50
            value: 25.46
            name: mR@50
          - type: R@50
            value: 35.68
            name: R@50
          - type: F1@50
            value: 29.72
            name: F1@50
          - type: mR@100
            value: 26.4
            name: mR@100
          - type: R@100
            value: 37.43
            name: R@100
          - type: F1@100
            value: 30.96
            name: F1@100
          - type: e2e_latency_ms
            value: 15.3
            name: e2e_latency_ms
---

# REACT++ Scene Graph Generation — PSG (yolo12l, yolo12m, yolo12s, yolo12n, yolov8m)

This repository contains **REACT++** model checkpoints for scene graph generation (SGG)
on the **PSG** benchmark, across 5 backbone sizes.

REACT++ is a parameter-efficient, attention-augmented relation predictor built on top of
a YOLO12 backbone.  It uses:

- **DAMP** (Detection-Anchored Multi-Scale Pooling), a new simple pooling algorithm for one-stage object detectors such as YOLO
- **SwiGLU gated MLP** for all feed-forward blocks (½ the params of ReLU-MLP at equal capacity)
- **Visual x Semantic cross-attention** — visual tokens attend to GloVe prototype embeddings
- **Geometry RoPE** — box-position encoded as a rotary frequency bias on the Q matrix
- **Prototype Momentum Buffer** — per-class EMA prototype bank
- **P5 Scene Context** — AIFI-enhanced P5 tokens provide global context via cross-attention

The models were trained with the
[SGG-Benchmark](https://github.com/Maelic/SGG-Benchmark) framework and described in the
[REACT++ paper (Neau et al., 2026)](https://arxiv.org/abs/2603.06386).

---

## Results — SGDet on PSG test split (ONNX, CUDA)

> Metrics from end-to-end ONNX evaluation (`tools/eval_onnx_psg.py`). E2E Latency = image load + pre-process + ONNX forward.

| Backbone | Params | R@20 | R@50 | R@100 | mR@20 | mR@50 | mR@100 | F1@20 | F1@50 | F1@100 | E2E Lat. (ms) |
|----------|:------:|-----:|-----:|------:|------:|------:|-------:|------:|------:|-------:|--------------:|
| yolo12l | ~26.5M | 30.99 | 35.3 | 36.68 | 23.2 | 25.49 | 26.45 | 26.53 | 29.6 | 30.74 | 19.6 |
| yolo12m | ~20.2M | 32.69 | 37.2 | 38.58 | 22.74 | 25.21 | 26.08 | 26.82 | 30.05 | 31.12 | 15.7 |
| yolo12s | ~9.2M | 29.28 | 33.48 | 34.74 | 21.12 | 23.21 | 23.77 | 24.54 | 27.41 | 28.23 | 12.2 |
| yolo12n | ~2.6M | 26.88 | 30.61 | 31.8 | 16.88 | 18.65 | 19.5 | 20.74 | 23.17 | 24.17 | 11.4 |
| yolov8m | ~25.9M | 30.69 | 35.68 | 37.43 | 22.75 | 25.46 | 26.4 | 26.13 | 29.72 | 30.96 | 15.3 |

---

## Checkpoints

| Variant | Sub-folder | Checkpoint files |
|---------|------------|-----------------|
| yolo12l | `yolo12l/` | `yolo12l/model.onnx` (ONNX) · `yolo12l/best_model_epoch_9.pth` (PyTorch) |
| yolo12m | `yolo12m/` | `yolo12m/model.onnx` (ONNX) · `yolo12m/best_model_epoch_9.pth` (PyTorch) |
| yolo12s | `yolo12s/` | `yolo12s/model.onnx` (ONNX) · `yolo12s/best_model_epoch_6.pth` (PyTorch) |
| yolo12n | `yolo12n/` | `yolo12n/model.onnx` (ONNX) · `yolo12n/best_model_epoch_5.pth` (PyTorch) |
| yolov8m | `yolov8m/` | `yolov8m/model.onnx` (ONNX) · `yolov8m/best_model_epoch_6.pth` (PyTorch) |

---

## Usage

### ONNX (recommended — no Python dependencies beyond onnxruntime)

```python
from huggingface_hub import hf_hub_download

onnx_path = hf_hub_download(
    repo_id="maelic/REACTPlusPlus_PSG",
    filename="yolo12l/react_pp_yolo12m.onnx",
    repo_type="model",
)
# Run with tools/eval_onnx_psg.py or load directly via onnxruntime
```

### PyTorch

```python
# 1. Clone the repository
#    git clone https://github.com/Maelic/SGG-Benchmark

# 2. Install dependencies
#    pip install -e .

# 3. Download checkpoint + config
from huggingface_hub import hf_hub_download

ckpt_path = hf_hub_download(
    repo_id="maelic/REACTPlusPlus_PSG",
    filename="yolo12l/best_model.pth",
    repo_type="model",
)
cfg_path = hf_hub_download(
    repo_id="maelic/REACTPlusPlus_PSG",
    filename="yolo12l/config.yml",
    repo_type="model",
)

# 4. Run evaluation
import subprocess
subprocess.run([
    "python", "tools/relation_eval_hydra.py",
    "--config-path", str(cfg_path),
    "--task", "sgdet",
    "--eval-only",
    "--checkpoint", str(ckpt_path),
])
```

---

## Citation

```bibtex
@article{neau2026reactpp,
  title   = {REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation
},
  author  = {Neau, Maëlic and Falomir, Zoe},
  year    = {2026},
  url     = {https://arxiv.org/abs/2603.06386},
}
```