---
language:
- en
tags:
- autonomous-vehicles
- danger-anticipation
- transformer
- computer-vision
- safety
license: mit
---

# PRECOG-HERALD: Proactive Danger Anticipation

**Author:** Nikhil Upadhyay | MSc Business Analytics | Dublin Business School
**Project:** [PRECOG-AV](https://github.com/TrazeMaG/PRECOG-AV)

## Overview

HERALD v2 anticipates danger in driving scenes from ViT-B/16 camera features.
Trained on 276,445 clips across 20 countries — the largest scale in accident anticipation research.
Module 2 of the PRECOG system.

## Results

### PhysicalAI-AV (25 countries)

| Metric | Value |
|--------|-------|
| Test AUC | **0.8805** |
| Average Precision | **0.2593** |
| Geographic Gap (GGG) | **-0.018** (better on unseen countries) |

### Standard Benchmarks

| Benchmark | Metric | PRECOG | Previous Best |
|-----------|--------|--------|---------------|
| CCD | AP | **99.95%** | 99.80% (RARE) |
| CCD | mTTA | **4.25s** | — |
| DAD | mTTA | **3.83s** | 3.16s (LATTE) |

Runs at **30 FPS** on a single RTX 4060 — 3x faster than real-time AV requirements.

## Usage

```python
import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download

class HERALDv2(nn.Module):
    def __init__(self, n_frames=5):
        super().__init__()
        self.cls_token   = nn.Parameter(torch.randn(1,1,768))
        self.pos_embed   = nn.Embedding(n_frames+1, 768)
        layer = nn.TransformerEncoderLayer(
            d_model=768, nhead=4, dim_feedforward=1536,
            dropout=0.3, batch_first=True, norm_first=True)
        self.transformer = nn.TransformerEncoder(layer, num_layers=2)
        self.cam_norm    = nn.LayerNorm(768)
        self.obj_encoder = nn.Sequential(
            nn.Linear(7,64), nn.GELU(), nn.Dropout(0.3), nn.Linear(64,128), nn.GELU())
        self.head = nn.Sequential(
            nn.Linear(896,256), nn.GELU(), nn.Dropout(0.3),
            nn.Linear(256,64), nn.GELU(), nn.Linear(64,1))
    def forward(self, x, obj):
        B   = x.shape[0]
        cls = self.cls_token.expand(B,-1,-1)
        x   = torch.cat([cls,x],dim=1)
        pos = torch.arange(x.shape[1], device=x.device)
        x   = x + self.pos_embed(pos)
        x   = self.cam_norm(self.transformer(x))
        return self.head(torch.cat([x[:,0], self.obj_encoder(obj)],dim=1)).squeeze(-1)

path = hf_hub_download("Trazemag/PRECOG-HERALD", "herald_v2_best.pt")
model = HERALDv2()
model.load_state_dict(torch.load(path, map_location="cpu"))
model.eval()
# x:   (1, N_FRAMES, 768) ViT-B/16 features
# obj: (1, 7) object proximity stats — pass zeros for camera-only mode
```

## Citation

```bibtex
@misc{upadhyay2026precog,
  title  = {PRECOG: Proactive Risk and Environmental Cognition for Autonomous Vehicles},
  author = {Upadhyay, Nikhil},
  year   = {2026},
  url    = {https://github.com/TrazeMaG/PRECOG-AV}
}
```