Buckets:

sillymonkey98
/

BADAS-Open-bucket

Files

xet

sillymonkey98/BADAS-Open-bucket / README.md

sillymonkey98

8 days ago

preview code

download

raw

8.59 kB

---
license: apache-2.0
gated: true
tags:
- video-classification
- collision-prediction
- ego-centric
- autonomous-driving
- adas
- dashcam
- v-jepa2
library_name: pytorch
pipeline_tag: video-classification
extra_gated_fields:
  First Name: text
  Last Name: text
  Title: text
  Company Email: text
  Company: text
  Topic of Interest:
    type: select
    options:
    - Research
    - Commercial
    - Education
    - Other
  I agree to use this model for lawful purposes only and acknowledge the terms and conditions: checkbox
extra_gated_prompt: >-
  By accessing this model, you agree to use it responsibly and in compliance
  with applicable laws. This model is intended for research and development
  purposes in driver assistance systems.
datasets:
- nexar-ai/nexar_collision_prediction
base_model:
- facebook/vjepa2-vitl-fpc16-256-ssv2
---
# BADAS-Open: Ego-Centric Collision Prediction Model

<div align="center">
  <img src="https://img.shields.io/badge/State--of--the--Art-Collision%20Prediction-blue" alt="SOTA">
  <img src="https://img.shields.io/badge/V--JEPA2-Foundation%20Model-green" alt="V-JEPA2">
  <img src="https://img.shields.io/badge/License-Apache%202.0-orange" alt="License">
</div>

## 🎯 Overview

**BADAS-Open** (V-JEPA2 Based Advanced Driver Assistance System) is a state-of-the-art collision prediction model specifically designed for **ego-centric** threat detection in real-world driving scenarios. Unlike traditional methods that detect any visible accident, BADAS focuses exclusively on collisions and near-misses that directly threaten the recording vehicle, dramatically reducing false alarms in real-world deployment.

<img src="assets/BADAS_Demo.gif" alt="BADAS Collision Prediction Example" width="750">

### Key Features

- 🎯 **Ego-Centric Focus**: Distinguishes between ego-vehicle threats and irrelevant accidents
- 🚗 **Real-World Trained**: Trained on 1,500 real dashcam videos from actual driving scenarios
- ⚡ **State-of-the-Art Performance**: Outperforms academic methods and commercial ADAS systems
- 🧠 **Foundation Model Based**: Built on V-JEPA2 for superior temporal understanding
- 🎬 **Near-Miss Learning**: Includes emergency maneuver scenarios for richer training signals

## 📊 Performance

BADAS-Open achieves state-of-the-art results across all major benchmarks when evaluated on ego-vehicle involved collisions:

<p align="center">
  <img src="assets/performance.png" alt="Performance Comparison" width="600">
</p>

| Dataset | AP ↑ | AUC ↑ | mTTA (s) ↓ | Test Size |
|---------|------|-------|------------|-----------|
| **Nexar** | **0.86** | **0.88** | 4.9 | 1,344 |
| **DoTA** | **0.94** | **0.70** | 4.0 | 367 |
| **DADA-2000** | **0.87** | **0.77** | 4.3 | 113 |
| **DAD** | **0.66** | **0.87** | 2.7 | 116 |

*Compared to baseline methods (DSTA, UString) with AP scores of 0.06-0.53*

### What Makes This Different?

Traditional collision prediction models are trained to detect **any accident in the camera's view**, leading to excessive false alarms from irrelevant incidents (e.g., accidents in adjacent lanes). BADAS solves this by:

1. **Ego-Centric Reformulation**: Only predicting collisions that directly threaten the ego vehicle
2. **Real-World Data**: Trained on actual dashcam footage, not synthetic or staged scenarios  
3. **Consensus-Based Timing**: Alert times validated by 10 certified defensive driving experts
4. **Near-Miss Inclusion**: Learning from successfully-avoided dangerous situations

<p align="center">
  <img src="assets/example.png" alt="Prediction Example" width="750">
  <br>
  <em>Example: BADAS prediction on real dashcam footage</em>
</p>

## 🚀 Quick Start

### Installation
```bash
pip install torch torchvision transformers huggingface_hub opencv-python numpy pillow albumentations
```

### Basic Usage
```python
from huggingface_hub import hf_hub_download
import sys
import os
# Download model loader
loader_path = hf_hub_download(
    repo_id="nexar-ai/badas-open",
    filename="badas_loader.py"
)
sys.path.insert(0, os.path.dirname(loader_path))
# Load model
from badas_loader import load_badas_model
model = load_badas_model()
# Predict on video
predictions = model.predict("dashcam_video.mp4")
# Get collision probability over time
for frame_idx, prob in enumerate(predictions):
    if prob > 0.8:  # High risk threshold
        print(f"⚠️ Collision risk at frame {frame_idx}: {prob:.2%}")
```

## 🏗️ Model Architecture
```
Input Video (16 frames @ 256×256)
    ↓
V-JEPA2 Encoder (ViT-L)
    ↓ (2048 patches × 1024 dim)
Attentive Probe Aggregation (12 queries × 64 dim)
    ↓
3-Layer MLP Prediction Head
    ↓
Collision Probability [0, 1]
```

## 🏗️ Model Architecture

<p align="center">
  <img src="assets/arch.png" alt="BADAS Architecture" width="650">
  <br>
  <em>BADAS architecture: V-JEPA2 backbone with attentive probe aggregation and MLP head</em>
</p>

**Key Components:**
- **Backbone**: V-JEPA2 (Vision Joint-Embedding Predictive Architecture v2)
- **Patch Aggregation**: Attentive probe with 12 learned queries
- **Head**: 3-layer MLP with GELU activation, LayerNorm, dropout 0.1
- **Training**: End-to-end fine-tuning on Nexar dataset (1.5k videos)

## 📋 Requirements

- Python ≥ 3.8
- PyTorch ≥ 2.0
- torchvision ≥ 0.15
- transformers ≥ 4.30
- opencv-python ≥ 4.8
- huggingface_hub ≥ 0.16
## 📂 Model Files
```
badas-open/
├── model.safetensors         # Model weights
├── config.json               # Model configuration  
├── badas_loader.py          # Loading utilities
├── preprocessing.py          # Video preprocessing
└── README.md                # This file
```
## 🔬 Training Details
- **Dataset**: Nexar Dashcam Collision Prediction Dataset (1,500 videos)
  - 750 collision/near-miss events
  - 750 normal driving samples
- **Optimizer**: AdamW (lr=1e-5, weight decay=1e-4)
- **Loss**: Binary Cross-Entropy
- **Augmentations**: Weather simulation (rain, snow), lighting variations, motion blur
- **Training Time**: ~8 hours on 4× A100 GPUs
## 🎯 Use Cases
### ✅ Recommended Uses
- Driver assistance system research
- Autonomous vehicle safety testing
- Dashcam collision warning applications
- Traffic safety analysis
- Dataset for training improved models
### ⚠️ Limitations
- **Long-tail events**: Performance drops on rare categories (animals, motorcycles)
- **Monocular only**: Requires single-camera dashcam input
- **Processing delay**: 16-frame buffer introduces latency
- **Not for critical safety**: Research model, not certified for production deployment
## 📖 Citation
If you use this model in your research, please cite:
```bibtex
@article{goldshmidt2025badas,
  title={BADAS: Context Aware Collision Prediction Using Real-World Dashcam Data},
  author={Goldshmidt, Roni and Scott, Hamish and Niccolini, Lorenzo and 
          Zhu, Shizhan and Moura, Daniel and Zvitia, Orly},
  journal={arXiv preprint},
  year={2025}
}
```
## 🔗 Resources

- 📄 **Paper**: [arXiv](https://arxiv.org) *(coming soon)*
- 🗂️ **Dataset**: [Nexar Collision Prediction Dataset](https://huggingface.co/datasets/nexar-ai/nexar_collision_prediction)
- 🏆 **Challenge**: [Kaggle Competition](https://www.kaggle.com/competitions/nexar-collision-prediction)
- 🌐 **Website**: [www.nexar-ai.com](https://www.nexar-ai.com)
- 💻 **Code**: [GitHub Repository](https://github.com/getnexar/BADAS-Open)

## 🤝 Model Variants

- **BADAS-Open** (this model): Trained on 1.5k public videos
- **BADAS-Pro**: Commercial variant trained on 40k proprietary videos
  - Higher performance (AP: 0.91 on Nexar)
  - Better edge-case handling
  - Contact Nexar for licensing

## ⚖️ License & Terms

This model is released under **Apache 2.0 License** with the following conditions:

1. ✅ Free for research and commercial use
2. ✅ Modification and redistribution permitted
3. ⚠️ No warranty provided - use at your own risk
4. ⚠️ Not certified for safety-critical applications
5. 📝 Must provide attribution when using

**Responsible AI Notice**: This model is intended to assist human drivers, not replace them. Always maintain full attention while driving.

## 🙏 Acknowledgments

- V-JEPA2 foundation model by Meta AI Research
- Nexar's driver community for dataset contribution  
- Re-annotations of DAD, DADA-2000, and DoTA benchmarks

---

<div align="center">
  <b>Questions or Issues?</b><br>
  Open an issue on our <a href="https://github.com/nexar-ai/badas/issues">GitHub</a> or contact us at research@nexar.com
</div>

Xet Storage Details

Size:: 8.59 kB
Xet hash:: 9dca458bde01c128f0dd09b19ab45f05ed25f3ede0ffcb5a760c749beafd669d

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	---
	license: apache-2.0
	gated: true
	tags:
	- video-classification
	- collision-prediction
	- ego-centric
	- autonomous-driving
	- adas
	- dashcam
	- v-jepa2
	library_name: pytorch
	pipeline_tag: video-classification
	extra_gated_fields:
	First Name: text
	Last Name: text
	Title: text
	Company Email: text
	Company: text
	Topic of Interest:
	type: select
	options:
	- Research
	- Commercial
	- Education
	- Other
	I agree to use this model for lawful purposes only and acknowledge the terms and conditions: checkbox
	extra_gated_prompt: >-
	By accessing this model, you agree to use it responsibly and in compliance
	with applicable laws. This model is intended for research and development
	purposes in driver assistance systems.
	datasets:
	- nexar-ai/nexar_collision_prediction
	base_model:
	- facebook/vjepa2-vitl-fpc16-256-ssv2
	---

	# BADAS-Open: Ego-Centric Collision Prediction Model

	<div align="center">
	<img src="https://img.shields.io/badge/State--of--the--Art-Collision%20Prediction-blue" alt="SOTA">
	<img src="https://img.shields.io/badge/V--JEPA2-Foundation%20Model-green" alt="V-JEPA2">
	<img src="https://img.shields.io/badge/License-Apache%202.0-orange" alt="License">
	</div>

	## 🎯 Overview

	BADAS-Open (V-JEPA2 Based Advanced Driver Assistance System) is a state-of-the-art collision prediction model specifically designed for ego-centric threat detection in real-world driving scenarios. Unlike traditional methods that detect any visible accident, BADAS focuses exclusively on collisions and near-misses that directly threaten the recording vehicle, dramatically reducing false alarms in real-world deployment.

	<img src="assets/BADAS_Demo.gif" alt="BADAS Collision Prediction Example" width="750">

	### Key Features

	- 🎯 Ego-Centric Focus: Distinguishes between ego-vehicle threats and irrelevant accidents
	- 🚗 Real-World Trained: Trained on 1,500 real dashcam videos from actual driving scenarios
	- ⚡ State-of-the-Art Performance: Outperforms academic methods and commercial ADAS systems
	- 🧠 Foundation Model Based: Built on V-JEPA2 for superior temporal understanding
	- 🎬 Near-Miss Learning: Includes emergency maneuver scenarios for richer training signals

	## 📊 Performance

	BADAS-Open achieves state-of-the-art results across all major benchmarks when evaluated on ego-vehicle involved collisions:

	<p align="center">
	<img src="assets/performance.png" alt="Performance Comparison" width="600">
	</p>

	\| Dataset \| AP ↑ \| AUC ↑ \| mTTA (s) ↓ \| Test Size \|
	\|---------\|------\|-------\|------------\|-----------\|
	\| Nexar \| 0.86 \| 0.88 \| 4.9 \| 1,344 \|
	\| DoTA \| 0.94 \| 0.70 \| 4.0 \| 367 \|
	\| DADA-2000 \| 0.87 \| 0.77 \| 4.3 \| 113 \|
	\| DAD \| 0.66 \| 0.87 \| 2.7 \| 116 \|

	Compared to baseline methods (DSTA, UString) with AP scores of 0.06-0.53

	### What Makes This Different?

	Traditional collision prediction models are trained to detect any accident in the camera's view, leading to excessive false alarms from irrelevant incidents (e.g., accidents in adjacent lanes). BADAS solves this by:

	1. Ego-Centric Reformulation: Only predicting collisions that directly threaten the ego vehicle
	2. Real-World Data: Trained on actual dashcam footage, not synthetic or staged scenarios
	3. Consensus-Based Timing: Alert times validated by 10 certified defensive driving experts
	4. Near-Miss Inclusion: Learning from successfully-avoided dangerous situations

	<p align="center">
	<img src="assets/example.png" alt="Prediction Example" width="750">
	<br>
	<em>Example: BADAS prediction on real dashcam footage</em>
	</p>

	## 🚀 Quick Start

	### Installation
	```bash
	pip install torch torchvision transformers huggingface_hub opencv-python numpy pillow albumentations
	```

	### Basic Usage
	```python
	from huggingface_hub import hf_hub_download
	import sys
	import os

	# Download model loader
	loader_path = hf_hub_download(
	repo_id="nexar-ai/badas-open",
	filename="badas_loader.py"
	)
	sys.path.insert(0, os.path.dirname(loader_path))

	# Load model
	from badas_loader import load_badas_model
	model = load_badas_model()

	# Predict on video
	predictions = model.predict("dashcam_video.mp4")

	# Get collision probability over time
	for frame_idx, prob in enumerate(predictions):
	if prob > 0.8: # High risk threshold
	print(f"⚠️ Collision risk at frame {frame_idx}: {prob:.2%}")
	```

	## 🏗️ Model Architecture
	```
	Input Video (16 frames @ 256×256)
	↓
	V-JEPA2 Encoder (ViT-L)
	↓ (2048 patches × 1024 dim)
	Attentive Probe Aggregation (12 queries × 64 dim)
	↓
	3-Layer MLP Prediction Head
	↓
	Collision Probability [0, 1]
	```

	## 🏗️ Model Architecture

	<p align="center">
	<img src="assets/arch.png" alt="BADAS Architecture" width="650">
	<br>
	<em>BADAS architecture: V-JEPA2 backbone with attentive probe aggregation and MLP head</em>
	</p>

	Key Components:
	- Backbone: V-JEPA2 (Vision Joint-Embedding Predictive Architecture v2)
	- Patch Aggregation: Attentive probe with 12 learned queries
	- Head: 3-layer MLP with GELU activation, LayerNorm, dropout 0.1
	- Training: End-to-end fine-tuning on Nexar dataset (1.5k videos)

	## 📋 Requirements

	- Python ≥ 3.8
	- PyTorch ≥ 2.0
	- torchvision ≥ 0.15
	- transformers ≥ 4.30
	- opencv-python ≥ 4.8
	- huggingface_hub ≥ 0.16

	## 📂 Model Files
	```
	badas-open/
	├── model.safetensors # Model weights
	├── config.json # Model configuration
	├── badas_loader.py # Loading utilities
	├── preprocessing.py # Video preprocessing
	└── README.md # This file
	```

	## 🔬 Training Details

	- Dataset: Nexar Dashcam Collision Prediction Dataset (1,500 videos)
	- 750 collision/near-miss events
	- 750 normal driving samples
	- Optimizer: AdamW (lr=1e-5, weight decay=1e-4)
	- Loss: Binary Cross-Entropy
	- Augmentations: Weather simulation (rain, snow), lighting variations, motion blur
	- Training Time: ~8 hours on 4× A100 GPUs

	## 🎯 Use Cases

	### ✅ Recommended Uses
	- Driver assistance system research
	- Autonomous vehicle safety testing
	- Dashcam collision warning applications
	- Traffic safety analysis
	- Dataset for training improved models

	### ⚠️ Limitations
	- Long-tail events: Performance drops on rare categories (animals, motorcycles)
	- Monocular only: Requires single-camera dashcam input
	- Processing delay: 16-frame buffer introduces latency
	- Not for critical safety: Research model, not certified for production deployment

	## 📖 Citation

	If you use this model in your research, please cite:
	```bibtex
	@article{goldshmidt2025badas,
	title={BADAS: Context Aware Collision Prediction Using Real-World Dashcam Data},
	author={Goldshmidt, Roni and Scott, Hamish and Niccolini, Lorenzo and
	Zhu, Shizhan and Moura, Daniel and Zvitia, Orly},
	journal={arXiv preprint},
	year={2025}
	}
	```

	## 🔗 Resources

	- 📄 Paper: [arXiv](https://arxiv.org) (coming soon)
	- 🗂️ Dataset: [Nexar Collision Prediction Dataset](https://huggingface.co/datasets/nexar-ai/nexar_collision_prediction)
	- 🏆 Challenge: [Kaggle Competition](https://www.kaggle.com/competitions/nexar-collision-prediction)
	- 🌐 Website: [www.nexar-ai.com](https://www.nexar-ai.com)
	- 💻 Code: [GitHub Repository](https://github.com/getnexar/BADAS-Open)

	## 🤝 Model Variants

	- BADAS-Open (this model): Trained on 1.5k public videos
	- BADAS-Pro: Commercial variant trained on 40k proprietary videos
	- Higher performance (AP: 0.91 on Nexar)
	- Better edge-case handling
	- Contact Nexar for licensing

	## ⚖️ License & Terms

	This model is released under Apache 2.0 License with the following conditions:

	1. ✅ Free for research and commercial use
	2. ✅ Modification and redistribution permitted
	3. ⚠️ No warranty provided - use at your own risk
	4. ⚠️ Not certified for safety-critical applications
	5. 📝 Must provide attribution when using

	Responsible AI Notice: This model is intended to assist human drivers, not replace them. Always maintain full attention while driving.

	## 🙏 Acknowledgments

	- V-JEPA2 foundation model by Meta AI Research
	- Nexar's driver community for dataset contribution
	- Re-annotations of DAD, DADA-2000, and DoTA benchmarks

	---

	<div align="center">
	<b>Questions or Issues?</b><br>
	Open an issue on our <a href="https://github.com/nexar-ai/badas/issues">GitHub</a> or contact us at research@nexar.com
	</div>