insailab commited on
Commit
59dd0af
Β·
verified Β·
1 Parent(s): fdb0f39

docs: add ISSAFE model card

Browse files
Files changed (1) hide show
  1. README.md +107 -0
README.md ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: pytorch
4
+ tags:
5
+ - multimodal
6
+ - semantic-segmentation
7
+ - vision
8
+ - event-camera
9
+ - accident-scene
10
+ - iros-2021
11
+ - tits-2021
12
+ - edcnet
13
+ - issafe
14
+ pipeline_tag: image-segmentation
15
+ ---
16
+
17
+ # ISSAFE & EDCNet: Event-Driven Scene Segmentation in Accidents
18
+
19
+ [IROS 2021 & IEEE T-ITS 2021] Official Model Checkpoints Repository for **ISSAFE** and **EDCNet**.
20
+
21
+ - **GitHub Repository:** https://github.com/InSAI-Lab/ISSAFE
22
+ - **Dataset Repository:** [InSAI-Lab/DADA-seg](https://huggingface.co/datasets/InSAI-Lab/DADA-seg)
23
+ - **Collection:** [ISSAFE & EDCNet (IROS 2021 & T-ITS 2021)](https://huggingface.co/collections/InSAI-Lab/issafe-and-edcnet-iros-2021-and-t-its-2021-6ad1b2429101a4c06509e4cb)
24
+ - **Paper (IROS 2021):** [ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data](https://arxiv.org/abs/2008.08974)
25
+ - **Paper (IEEE T-ITS 2021):** [Exploring Event-Driven Dynamic Context for Accident Scene Segmentation](https://arxiv.org/abs/2112.05006)
26
+
27
+ ---
28
+
29
+ ## Model Description
30
+
31
+ **EDCNet** (Event-Driven dynamic Context Network) and **ISSAFE** address semantic segmentation in extreme traffic scenarios and accident scenes. By leveraging event cameras with microsecond temporal resolution, the architecture extracts dynamic motion cues to compensate for motion blur, severe exposure shifts, and sudden collisions where traditional static RGB cameras fail.
32
+
33
+ Key Features:
34
+ - **Asymmetric Multimodal Architecture:** Combines an RGB stream and an Event Volume stream.
35
+ - **Dynamic Context Mining:** Preserves fine-grained motion of crashing foreground objects.
36
+ - **Domain-to-Scene (D2S) Adaptation:** Trained on synthetic/real event volumes and generalized to critical accident scenarios.
37
+
38
+ ## Checkpoints Organization
39
+
40
+ The repository provides trained model weights matching the official ISSAFE codebase evaluation structure:
41
+
42
+ ```text
43
+ β”œβ”€β”€ run/
44
+ β”‚ └── cityscapesevent/
45
+ β”‚ └── test_EDCNet_r18/
46
+ β”‚ └── model_best.pth # EDCNet (ResNet-18, B=2 event time bins)
47
+ └── test_EDCNet_r18/
48
+ └── model_best.pth # Convenience direct path copy
49
+ ```
50
+
51
+ ## Quick Start & Evaluation
52
+
53
+ ### 1. Download Model Weights
54
+
55
+ Using Hugging Face CLI:
56
+
57
+ ```bash
58
+ # Download model weight to the local workspace
59
+ hf download InSAI-Lab/ISSAFE run/cityscapesevent/test_EDCNet_r18/model_best.pth --local-dir .
60
+ ```
61
+
62
+ Using Python (`huggingface_hub`):
63
+
64
+ ```python
65
+ from huggingface_hub import hf_hub_download
66
+
67
+ model_path = hf_hub_download(
68
+ repo_id="InSAI-Lab/ISSAFE",
69
+ filename="run/cityscapesevent/test_EDCNet_r18/model_best.pth",
70
+ local_dir="."
71
+ )
72
+ ```
73
+
74
+ ### 2. Evaluation
75
+
76
+ In the [ISSAFE repository](https://github.com/InSAI-Lab/ISSAFE):
77
+
78
+ ```bash
79
+ # Evaluate EDCNet on DADA-seg validation set
80
+ python eval.py --dataset dadaevent --weight-path run/cityscapesevent/test_EDCNet_r18/model_best.pth
81
+ ```
82
+
83
+ ## Citation
84
+
85
+ ```bibtex
86
+ @inproceedings{zhang2021issafe,
87
+ author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
88
+ booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
89
+ title={ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data},
90
+ year={2021},
91
+ pages={1132-1139},
92
+ doi={10.1109/IROS51168.2021.9636109}
93
+ }
94
+
95
+ @article{zhang2021edcnet,
96
+ author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
97
+ journal={IEEE Transactions on Intelligent Transportation Systems},
98
+ title={Exploring Event-Driven Dynamic Context for Accident Scene Segmentation},
99
+ year={2021},
100
+ pages={1-17},
101
+ doi={10.1109/TITS.2021.3134828}
102
+ }
103
+ ```
104
+
105
+ ## License
106
+
107
+ This repository is licensed under the [Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).