maelic commited on
Commit
68ffc85
·
verified ·
1 Parent(s): 98ed338

Update model card

Browse files
Files changed (1) hide show
  1. README.md +228 -0
README.md ADDED
@@ -0,0 +1,228 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - scene-graph-generation
7
+ - object-detection
8
+ - visual-relationship-detection
9
+ - pytorch
10
+ - yolo
11
+ pipeline_tag: object-detection
12
+ library_name: sgg-benchmark
13
+ model-index:
14
+ - name: REACT++ yolo12m
15
+ results:
16
+ - task:
17
+ type: object-detection
18
+ name: Scene Graph Detection
19
+ dataset:
20
+ name: VG150
21
+ type: vg150
22
+ metrics:
23
+ - type: mR@20
24
+ value: 10.52
25
+ name: mR@20
26
+ - type: R@20
27
+ value: 18.32
28
+ name: R@20
29
+ - type: F1@20
30
+ value: 13.36
31
+ name: F1@20
32
+ - type: mR@50
33
+ value: 13.22
34
+ name: mR@50
35
+ - type: R@50
36
+ value: 22.54
37
+ name: R@50
38
+ - type: F1@50
39
+ value: 16.67
40
+ name: F1@50
41
+ - type: mR@100
42
+ value: 13.96
43
+ name: mR@100
44
+ - type: R@100
45
+ value: 23.77
46
+ name: R@100
47
+ - type: F1@100
48
+ value: 17.59
49
+ name: F1@100
50
+ - type: e2e_latency_ms
51
+ value: 19.4
52
+ name: e2e_latency_ms
53
+ - name: REACT++ yolo26m
54
+ results:
55
+ - task:
56
+ type: object-detection
57
+ name: Scene Graph Detection
58
+ dataset:
59
+ name: VG150
60
+ type: vg150
61
+ metrics:
62
+ - type: mR@20
63
+ value: 10.32
64
+ name: mR@20
65
+ - type: R@20
66
+ value: 20.0
67
+ name: R@20
68
+ - type: mR@50
69
+ value: 13.94
70
+ name: mR@50
71
+ - type: R@50
72
+ value: 26.9
73
+ name: R@50
74
+ - type: mR@100
75
+ value: 16.48
76
+ name: mR@100
77
+ - type: R@100
78
+ value: 32.08
79
+ name: R@100
80
+ - type: mean_recall
81
+ value: 21.87
82
+ name: mean_recall
83
+ - name: REACT++ yolov8m
84
+ results:
85
+ - task:
86
+ type: object-detection
87
+ name: Scene Graph Detection
88
+ dataset:
89
+ name: VG150
90
+ type: vg150
91
+ metrics:
92
+ - type: mR@20
93
+ value: 12.05
94
+ name: mR@20
95
+ - type: R@20
96
+ value: 22.78
97
+ name: R@20
98
+ - type: F1@20
99
+ value: 15.76
100
+ name: F1@20
101
+ - type: mR@50
102
+ value: 15.42
103
+ name: mR@50
104
+ - type: R@50
105
+ value: 28.73
106
+ name: R@50
107
+ - type: F1@50
108
+ value: 20.07
109
+ name: F1@50
110
+ - type: mR@100
111
+ value: 16.51
112
+ name: mR@100
113
+ - type: R@100
114
+ value: 30.84
115
+ name: R@100
116
+ - type: F1@100
117
+ value: 21.51
118
+ name: F1@100
119
+ - type: e2e_latency_ms
120
+ value: 17.8
121
+ name: e2e_latency_ms
122
+ ---
123
+
124
+ # REACT++ Scene Graph Generation — VG150 (yolo12m, yolo26m, yolov8m)
125
+
126
+ This repository contains **REACT++** model checkpoints for scene graph generation (SGG)
127
+ on the **VG150** benchmark, across 3 backbone sizes.
128
+
129
+ REACT++ is a parameter-efficient, attention-augmented relation predictor built on top of
130
+ a YOLO backbone. It uses:
131
+
132
+ - **DAMP** (Detection-Anchored Multi-Scale Pooling), a new simple pooling algorithm for one-stage object detectors such as YOLO
133
+ - **SwiGLU gated MLP** for all feed-forward blocks (½ the params of ReLU-MLP at equal capacity)
134
+ - **Visual x Semantic cross-attention** — visual tokens attend to GloVe prototype embeddings
135
+ - **Geometry RoPE** — box-position encoded as a rotary frequency bias on the Q matrix
136
+ - **Prototype Momentum Buffer** — per-class EMA prototype bank
137
+ - **P5 Scene Context** — AIFI-enhanced P5 tokens provide global context via cross-attention
138
+
139
+ The models were trained with the
140
+ [SGG-Benchmark](https://github.com/Maelic/SGG-Benchmark) framework and described in the
141
+ [REACT++ paper (Neau et al., 2026)](https://arxiv.org/abs/2603.06386).
142
+
143
+ ---
144
+
145
+ ## Results — SGDet on VG150 test split (ONNX, CUDA)
146
+
147
+ > Metrics from end-to-end ONNX evaluation (`tools/eval_onnx_psg.py`). E2E Latency = image load + pre-process + ONNX forward.
148
+
149
+ | Backbone | Params | R@20 | R@50 | R@100 | mR@20 | mR@50 | mR@100 | F1@20 | F1@50 | F1@100 | E2E Lat. (ms) |
150
+ |----------|:------:|-----:|-----:|------:|------:|------:|-------:|------:|------:|-------:|--------------:|
151
+ | yolo12m | ~20.2M | 18.32 | 22.54 | 23.77 | 10.52 | 13.22 | 13.96 | 13.36 | 16.67 | 17.59 | 19.4 |
152
+ | yolo26m | ~20.2M | 20.0 | 26.9 | 32.08 | 10.32 | 13.94 | 16.48 | - | - | - | - |
153
+ | yolov8m | ~25.9M | 22.78 | 28.73 | 30.84 | 12.05 | 15.42 | 16.51 | 15.76 | 20.07 | 21.51 | 17.8 |
154
+
155
+ ---
156
+
157
+ ## Checkpoints
158
+
159
+ | Variant | Sub-folder | Checkpoint files |
160
+ |---------|------------|-----------------|
161
+ | yolo12m | `yolo12m/` | `yolo12m/model.onnx` (ONNX) · `yolo12m/best_model_epoch_19.pth` (PyTorch) |
162
+ | yolo26m | `yolo26m/` | `yolo26m/model.onnx` (ONNX) · `yolo26m/best_model_epoch_18.pth` (PyTorch) |
163
+ | yolov8m | `yolov8m/` | `yolov8m/model.onnx` (ONNX) · `yolov8m/best_model_epoch_6.pth` (PyTorch) |
164
+
165
+ ---
166
+
167
+ ## Usage
168
+
169
+ ### ONNX (recommended — no Python dependencies beyond onnxruntime)
170
+
171
+ ```python
172
+ from huggingface_hub import hf_hub_download
173
+
174
+ onnx_path = hf_hub_download(
175
+ repo_id="maelic/REACTPlusPlus_VG150",
176
+ filename="yolo12m/react_pp_yolo12m.onnx",
177
+ repo_type="model",
178
+ )
179
+ # Run with tools/eval_onnx_psg.py or load directly via onnxruntime
180
+ ```
181
+
182
+ ### PyTorch
183
+
184
+ ```python
185
+ # 1. Clone the repository
186
+ # git clone https://github.com/Maelic/SGG-Benchmark
187
+
188
+ # 2. Install dependencies
189
+ # pip install -e .
190
+
191
+ # 3. Download checkpoint + config
192
+ from huggingface_hub import hf_hub_download
193
+
194
+ ckpt_path = hf_hub_download(
195
+ repo_id="maelic/REACTPlusPlus_VG150",
196
+ filename="yolo12m/best_model.pth",
197
+ repo_type="model",
198
+ )
199
+ cfg_path = hf_hub_download(
200
+ repo_id="maelic/REACTPlusPlus_VG150",
201
+ filename="yolo12m/config.yml",
202
+ repo_type="model",
203
+ )
204
+
205
+ # 4. Run evaluation
206
+ import subprocess
207
+ subprocess.run([
208
+ "python", "tools/relation_eval_hydra.py",
209
+ "--config-path", str(cfg_path),
210
+ "--task", "sgdet",
211
+ "--eval-only",
212
+ "--checkpoint", str(ckpt_path),
213
+ ])
214
+ ```
215
+
216
+ ---
217
+
218
+ ## Citation
219
+
220
+ ```bibtex
221
+ @article{neau2026reactpp,
222
+ title = {REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation
223
+ },
224
+ author = {Neau, Maëlic and Falomir, Zoe},
225
+ year = {2026},
226
+ url = {https://arxiv.org/abs/2603.06386},
227
+ }
228
+ ```