File size: 3,690 Bytes
59dd0af | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | ---
license: apache-2.0
library_name: pytorch
tags:
- multimodal
- semantic-segmentation
- vision
- event-camera
- accident-scene
- iros-2021
- tits-2021
- edcnet
- issafe
pipeline_tag: image-segmentation
---
# ISSAFE & EDCNet: Event-Driven Scene Segmentation in Accidents
[IROS 2021 & IEEE T-ITS 2021] Official Model Checkpoints Repository for **ISSAFE** and **EDCNet**.
- **GitHub Repository:** https://github.com/InSAI-Lab/ISSAFE
- **Dataset Repository:** [InSAI-Lab/DADA-seg](https://huggingface.co/datasets/InSAI-Lab/DADA-seg)
- **Collection:** [ISSAFE & EDCNet (IROS 2021 & T-ITS 2021)](https://huggingface.co/collections/InSAI-Lab/issafe-and-edcnet-iros-2021-and-t-its-2021-6ad1b2429101a4c06509e4cb)
- **Paper (IROS 2021):** [ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data](https://arxiv.org/abs/2008.08974)
- **Paper (IEEE T-ITS 2021):** [Exploring Event-Driven Dynamic Context for Accident Scene Segmentation](https://arxiv.org/abs/2112.05006)
---
## Model Description
**EDCNet** (Event-Driven dynamic Context Network) and **ISSAFE** address semantic segmentation in extreme traffic scenarios and accident scenes. By leveraging event cameras with microsecond temporal resolution, the architecture extracts dynamic motion cues to compensate for motion blur, severe exposure shifts, and sudden collisions where traditional static RGB cameras fail.
Key Features:
- **Asymmetric Multimodal Architecture:** Combines an RGB stream and an Event Volume stream.
- **Dynamic Context Mining:** Preserves fine-grained motion of crashing foreground objects.
- **Domain-to-Scene (D2S) Adaptation:** Trained on synthetic/real event volumes and generalized to critical accident scenarios.
## Checkpoints Organization
The repository provides trained model weights matching the official ISSAFE codebase evaluation structure:
```text
├── run/
│ └── cityscapesevent/
│ └── test_EDCNet_r18/
│ └── model_best.pth # EDCNet (ResNet-18, B=2 event time bins)
└── test_EDCNet_r18/
└── model_best.pth # Convenience direct path copy
```
## Quick Start & Evaluation
### 1. Download Model Weights
Using Hugging Face CLI:
```bash
# Download model weight to the local workspace
hf download InSAI-Lab/ISSAFE run/cityscapesevent/test_EDCNet_r18/model_best.pth --local-dir .
```
Using Python (`huggingface_hub`):
```python
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(
repo_id="InSAI-Lab/ISSAFE",
filename="run/cityscapesevent/test_EDCNet_r18/model_best.pth",
local_dir="."
)
```
### 2. Evaluation
In the [ISSAFE repository](https://github.com/InSAI-Lab/ISSAFE):
```bash
# Evaluate EDCNet on DADA-seg validation set
python eval.py --dataset dadaevent --weight-path run/cityscapesevent/test_EDCNet_r18/model_best.pth
```
## Citation
```bibtex
@inproceedings{zhang2021issafe,
author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
title={ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data},
year={2021},
pages={1132-1139},
doi={10.1109/IROS51168.2021.9636109}
}
@article{zhang2021edcnet,
author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
journal={IEEE Transactions on Intelligent Transportation Systems},
title={Exploring Event-Driven Dynamic Context for Accident Scene Segmentation},
year={2021},
pages={1-17},
doi={10.1109/TITS.2021.3134828}
}
```
## License
This repository is licensed under the [Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
|