| --- |
| language: |
| - en |
| tags: |
| - autonomous-vehicles |
| - danger-anticipation |
| - transformer |
| - computer-vision |
| - safety |
| license: mit |
| --- |
| |
| # PRECOG-HERALD: Proactive Danger Anticipation |
|
|
| **Author:** Nikhil Upadhyay | MSc Business Analytics | Dublin Business School |
| **Project:** [PRECOG-AV](https://github.com/TrazeMaG/PRECOG-AV) |
|
|
| ## Overview |
|
|
| HERALD v2 anticipates danger in driving scenes from ViT-B/16 camera features. |
| Trained on 276,445 clips across 20 countries — the largest scale in accident anticipation research. |
| Module 2 of the PRECOG system. |
|
|
| ## Results |
|
|
| ### PhysicalAI-AV (25 countries) |
|
|
| | Metric | Value | |
| |--------|-------| |
| | Test AUC | **0.8805** | |
| | Average Precision | **0.2593** | |
| | Geographic Gap (GGG) | **-0.018** (better on unseen countries) | |
|
|
| ### Standard Benchmarks |
|
|
| | Benchmark | Metric | PRECOG | Previous Best | |
| |-----------|--------|--------|---------------| |
| | CCD | AP | **99.95%** | 99.80% (RARE) | |
| | CCD | mTTA | **4.25s** | — | |
| | DAD | mTTA | **3.83s** | 3.16s (LATTE) | |
|
|
| Runs at **30 FPS** on a single RTX 4060 — 3x faster than real-time AV requirements. |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| import torch.nn as nn |
| from huggingface_hub import hf_hub_download |
| |
| class HERALDv2(nn.Module): |
| def __init__(self, n_frames=5): |
| super().__init__() |
| self.cls_token = nn.Parameter(torch.randn(1,1,768)) |
| self.pos_embed = nn.Embedding(n_frames+1, 768) |
| layer = nn.TransformerEncoderLayer( |
| d_model=768, nhead=4, dim_feedforward=1536, |
| dropout=0.3, batch_first=True, norm_first=True) |
| self.transformer = nn.TransformerEncoder(layer, num_layers=2) |
| self.cam_norm = nn.LayerNorm(768) |
| self.obj_encoder = nn.Sequential( |
| nn.Linear(7,64), nn.GELU(), nn.Dropout(0.3), nn.Linear(64,128), nn.GELU()) |
| self.head = nn.Sequential( |
| nn.Linear(896,256), nn.GELU(), nn.Dropout(0.3), |
| nn.Linear(256,64), nn.GELU(), nn.Linear(64,1)) |
| def forward(self, x, obj): |
| B = x.shape[0] |
| cls = self.cls_token.expand(B,-1,-1) |
| x = torch.cat([cls,x],dim=1) |
| pos = torch.arange(x.shape[1], device=x.device) |
| x = x + self.pos_embed(pos) |
| x = self.cam_norm(self.transformer(x)) |
| return self.head(torch.cat([x[:,0], self.obj_encoder(obj)],dim=1)).squeeze(-1) |
| |
| path = hf_hub_download("Trazemag/PRECOG-HERALD", "herald_v2_best.pt") |
| model = HERALDv2() |
| model.load_state_dict(torch.load(path, map_location="cpu")) |
| model.eval() |
| # x: (1, N_FRAMES, 768) ViT-B/16 features |
| # obj: (1, 7) object proximity stats — pass zeros for camera-only mode |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{upadhyay2026precog, |
| title = {PRECOG: Proactive Risk and Environmental Cognition for Autonomous Vehicles}, |
| author = {Upadhyay, Nikhil}, |
| year = {2026}, |
| url = {https://github.com/TrazeMaG/PRECOG-AV} |
| } |
| ``` |
