maelic commited on
Commit
f9086c8
·
verified ·
1 Parent(s): de52dbf

Update model card

Browse files
Files changed (1) hide show
  1. README.md +125 -0
README.md ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - scene-graph-generation
7
+ - object-detection
8
+ - visual-relationship-detection
9
+ - pytorch
10
+ - yolo
11
+ pipeline_tag: object-detection
12
+ library_name: sgg-benchmark
13
+ model-index:
14
+ - name: REACT++ yolov8m
15
+ results:
16
+ - task:
17
+ type: object-detection
18
+ name: Scene Graph Detection
19
+ dataset:
20
+ name: IndoorVG
21
+ type: indoorvg
22
+ metrics: []
23
+ ---
24
+
25
+ # REACT++ Scene Graph Generation — IndoorVG (yolov8m)
26
+
27
+ This repository contains **REACT++** model checkpoints for scene graph generation (SGG)
28
+ on the **IndoorVG** benchmark, across 1 backbone size.
29
+
30
+ REACT++ is a parameter-efficient, attention-augmented relation predictor built on top of
31
+ a YOLO backbone. It uses:
32
+
33
+ - **DAMP** (Detection-Anchored Multi-Scale Pooling), a new simple pooling algorithm for one-stage object detectors such as YOLO
34
+ - **SwiGLU gated MLP** for all feed-forward blocks (½ the params of ReLU-MLP at equal capacity)
35
+ - **Visual x Semantic cross-attention** — visual tokens attend to GloVe prototype embeddings
36
+ - **Geometry RoPE** — box-position encoded as a rotary frequency bias on the Q matrix
37
+ - **Prototype Momentum Buffer** — per-class EMA prototype bank
38
+ - **P5 Scene Context** — AIFI-enhanced P5 tokens provide global context via cross-attention
39
+
40
+ The models were trained with the
41
+ [SGG-Benchmark](https://github.com/Maelic/SGG-Benchmark) framework and described in the
42
+ [REACT++ paper (Neau et al., 2026)](https://arxiv.org/abs/2603.06386).
43
+
44
+ ---
45
+
46
+ ## Results — SGDet on IndoorVG test split (ONNX, CUDA)
47
+
48
+ > Metrics from end-to-end ONNX evaluation (`tools/eval_onnx_psg.py`). E2E Latency = image load + pre-process + ONNX forward.
49
+
50
+ | Backbone | Params | R@20 | R@50 | R@100 | mR@20 | mR@50 | mR@100 | F1@20 | F1@50 | F1@100 | E2E Lat. (ms) |
51
+ |----------|:------:|-----:|-----:|------:|------:|------:|-------:|------:|------:|-------:|--------------:|
52
+ | yolov8m | ~25.9M | - | - | - | - | - | - | - | - | - | - |
53
+
54
+ ---
55
+
56
+ ## Checkpoints
57
+
58
+ | Variant | Sub-folder | Checkpoint files |
59
+ |---------|------------|-----------------|
60
+ | yolov8m | `yolov8m/` | `yolov8m/model.onnx` (ONNX) · `yolov8m/best_model_epoch_8.pth` (PyTorch) |
61
+
62
+ ---
63
+
64
+ ## Usage
65
+
66
+ ### ONNX (recommended — no Python dependencies beyond onnxruntime)
67
+
68
+ ```python
69
+ from huggingface_hub import hf_hub_download
70
+
71
+ onnx_path = hf_hub_download(
72
+ repo_id="maelic/REACTPlusPlus_IndoorVG",
73
+ filename="yolov8m/react_pp_yolo12m.onnx",
74
+ repo_type="model",
75
+ )
76
+ # Run with tools/eval_onnx_psg.py or load directly via onnxruntime
77
+ ```
78
+
79
+ ### PyTorch
80
+
81
+ ```python
82
+ # 1. Clone the repository
83
+ # git clone https://github.com/Maelic/SGG-Benchmark
84
+
85
+ # 2. Install dependencies
86
+ # pip install -e .
87
+
88
+ # 3. Download checkpoint + config
89
+ from huggingface_hub import hf_hub_download
90
+
91
+ ckpt_path = hf_hub_download(
92
+ repo_id="maelic/REACTPlusPlus_IndoorVG",
93
+ filename="yolov8m/best_model.pth",
94
+ repo_type="model",
95
+ )
96
+ cfg_path = hf_hub_download(
97
+ repo_id="maelic/REACTPlusPlus_IndoorVG",
98
+ filename="yolov8m/config.yml",
99
+ repo_type="model",
100
+ )
101
+
102
+ # 4. Run evaluation
103
+ import subprocess
104
+ subprocess.run([
105
+ "python", "tools/relation_eval_hydra.py",
106
+ "--config-path", str(cfg_path),
107
+ "--task", "sgdet",
108
+ "--eval-only",
109
+ "--checkpoint", str(ckpt_path),
110
+ ])
111
+ ```
112
+
113
+ ---
114
+
115
+ ## Citation
116
+
117
+ ```bibtex
118
+ @article{neau2026reactpp,
119
+ title = {REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation
120
+ },
121
+ author = {Neau, Maëlic and Falomir, Zoe},
122
+ year = {2026},
123
+ url = {https://arxiv.org/abs/2603.06386},
124
+ }
125
+ ```