ISSAFE / README.md

docs: add ISSAFE model card

59dd0af verified 2 days ago

3.69 kB

license: apache-2.0
library_name: pytorch
tags:
  - multimodal
  - semantic-segmentation
  - vision
  - event-camera
  - accident-scene
  - iros-2021
  - tits-2021
  - edcnet
  - issafe
pipeline_tag: image-segmentation

ISSAFE & EDCNet: Event-Driven Scene Segmentation in Accidents

[IROS 2021 & IEEE T-ITS 2021] Official Model Checkpoints Repository for ISSAFE and EDCNet.

GitHub Repository: https://github.com/InSAI-Lab/ISSAFE
Dataset Repository: InSAI-Lab/DADA-seg
Collection: ISSAFE & EDCNet (IROS 2021 & T-ITS 2021)
Paper (IROS 2021): ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data
Paper (IEEE T-ITS 2021): Exploring Event-Driven Dynamic Context for Accident Scene Segmentation

Model Description

EDCNet (Event-Driven dynamic Context Network) and ISSAFE address semantic segmentation in extreme traffic scenarios and accident scenes. By leveraging event cameras with microsecond temporal resolution, the architecture extracts dynamic motion cues to compensate for motion blur, severe exposure shifts, and sudden collisions where traditional static RGB cameras fail.

Key Features:

Asymmetric Multimodal Architecture: Combines an RGB stream and an Event Volume stream.
Dynamic Context Mining: Preserves fine-grained motion of crashing foreground objects.
Domain-to-Scene (D2S) Adaptation: Trained on synthetic/real event volumes and generalized to critical accident scenarios.

Checkpoints Organization

The repository provides trained model weights matching the official ISSAFE codebase evaluation structure:

├── run/
│   └── cityscapesevent/
│       └── test_EDCNet_r18/
│           └── model_best.pth   # EDCNet (ResNet-18, B=2 event time bins)
└── test_EDCNet_r18/
    └── model_best.pth           # Convenience direct path copy

Quick Start & Evaluation

1. Download Model Weights

Using Hugging Face CLI:

# Download model weight to the local workspace
hf download InSAI-Lab/ISSAFE run/cityscapesevent/test_EDCNet_r18/model_best.pth --local-dir .

Using Python (huggingface_hub):

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="InSAI-Lab/ISSAFE",
    filename="run/cityscapesevent/test_EDCNet_r18/model_best.pth",
    local_dir="."
)

2. Evaluation

In the ISSAFE repository:

# Evaluate EDCNet on DADA-seg validation set
python eval.py --dataset dadaevent --weight-path run/cityscapesevent/test_EDCNet_r18/model_best.pth

Citation

@inproceedings{zhang2021issafe,
  author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
  booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data}, 
  year={2021},
  pages={1132-1139},
  doi={10.1109/IROS51168.2021.9636109}
}

@article{zhang2021edcnet,
  author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
  journal={IEEE Transactions on Intelligent Transportation Systems}, 
  title={Exploring Event-Driven Dynamic Context for Accident Scene Segmentation}, 
  year={2021},
  pages={1-17},
  doi={10.1109/TITS.2021.3134828}
}

License

This repository is licensed under the Apache-2.0 License.