PRECOG-HERALD: Proactive Danger Anticipation

Author: Nikhil Upadhyay | MSc Business Analytics | Dublin Business School Project: PRECOG-AV

Overview

HERALD v2 anticipates danger in driving scenes from ViT-B/16 camera features. Trained on 276,445 clips across 20 countries — the largest scale in accident anticipation research. Module 2 of the PRECOG system.

Results

PhysicalAI-AV (25 countries)

Metric	Value
Test AUC	0.8805
Average Precision	0.2593
Geographic Gap (GGG)	-0.018 (better on unseen countries)

Standard Benchmarks

Benchmark	Metric	PRECOG	Previous Best
CCD	AP	99.95%	99.80% (RARE)
CCD	mTTA	4.25s	—
DAD	mTTA	3.83s	3.16s (LATTE)

Runs at 30 FPS on a single RTX 4060 — 3x faster than real-time AV requirements.

Usage

import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download

class HERALDv2(nn.Module):
    def __init__(self, n_frames=5):
        super().__init__()
        self.cls_token   = nn.Parameter(torch.randn(1,1,768))
        self.pos_embed   = nn.Embedding(n_frames+1, 768)
        layer = nn.TransformerEncoderLayer(
            d_model=768, nhead=4, dim_feedforward=1536,
            dropout=0.3, batch_first=True, norm_first=True)
        self.transformer = nn.TransformerEncoder(layer, num_layers=2)
        self.cam_norm    = nn.LayerNorm(768)
        self.obj_encoder = nn.Sequential(
            nn.Linear(7,64), nn.GELU(), nn.Dropout(0.3), nn.Linear(64,128), nn.GELU())
        self.head = nn.Sequential(
            nn.Linear(896,256), nn.GELU(), nn.Dropout(0.3),
            nn.Linear(256,64), nn.GELU(), nn.Linear(64,1))
    def forward(self, x, obj):
        B   = x.shape[0]
        cls = self.cls_token.expand(B,-1,-1)
        x   = torch.cat([cls,x],dim=1)
        pos = torch.arange(x.shape[1], device=x.device)
        x   = x + self.pos_embed(pos)
        x   = self.cam_norm(self.transformer(x))
        return self.head(torch.cat([x[:,0], self.obj_encoder(obj)],dim=1)).squeeze(-1)

path = hf_hub_download("Trazemag/PRECOG-HERALD", "herald_v2_best.pt")
model = HERALDv2()
model.load_state_dict(torch.load(path, map_location="cpu"))
model.eval()
# x:   (1, N_FRAMES, 768) ViT-B/16 features
# obj: (1, 7) object proximity stats — pass zeros for camera-only mode

Citation

@misc{upadhyay2026precog,
  title  = {PRECOG: Proactive Risk and Environmental Cognition for Autonomous Vehicles},
  author = {Upadhyay, Nikhil},
  year   = {2026},
  url    = {https://github.com/TrazeMaG/PRECOG-AV}
}