README.md

---
license: mit
tags:
- autonomous-driving
- cooperative-perception
- bird-eye-view
- uncertainty-estimation
- hypernetwork
- segmentation
- opv2v
datasets:
- gqk/opv2v
---

# Hyper-V2X: Hypernetworks for Estimating Epistemic and Aleatoric Uncertainty in Cooperative Bird’s-Eye-View Semantic Segmentation

<h3 style="text-align:center; font-weight:600; letter-spacing:0.5px;">
  IEEE IV 2026 Oral
</h3>

[![Project Page](https://img.shields.io/badge/Project-Page-blue.svg)](https://abhishekjagtap1.github.io/HyperV2X/)&nbsp;&nbsp;
[![arXiv](https://img.shields.io/badge/arXiv-Paper-red.svg)](https://arxiv.org/abs/2605.21309v1)&nbsp;&nbsp;
[![GitHub Repo](https://img.shields.io/badge/GitHub-Code-FFD700?logo=github)](https://github.com/abhishekjagtap1/Hyper-V2X)&nbsp;&nbsp;

This paper introduces Hyper-V2X, a hypernetwork-based framework for estimating both epistemic and aleatoric uncertainties in V2X-based perception.
Specifically, we propose 
1) Bayesian hypernetwork formulation for cooperative perception.
2) V2X context embedding: that conditions a Bayesian hypernetwork on fused multi-agent features.
3) Partial weight generation for Bayesian hypernetworks; that avoids generating the full set of model parameters, enabling efficient and scalable uncertainty estimation.

## Overview

Hyper-V2X is a cooperative perception model for autonomous driving that performs **Bird’s-Eye-View (BEV) semantic segmentation** with **uncertainty estimation** under communication constraints.

It leverages a **Bayesian hypernetwork** conditioned on fused multi-agent BEV features to generate stochastic decoder weights, enabling:

- Epistemic uncertainty estimation (model uncertainty)
- Aleatoric uncertainty estimation (data uncertainty)
- Robust BEV semantic segmentation in V2X settings
- Performance under limited communication bandwidth / compression constraints


## Training Data

Hyper-V2X is trained on:

- OPV2V cooperative perception dataset

Dataset reference:
- https://huggingface.co/datasets/gqk/opv2v



## Quick Start

For full installation instructions please refer to the [Hyper-V2X GitHub repository](https://github.com/abhishekjagtap1/Hyper-V2X).

Once the dependencies are installed, you can use the load the checkpoints from Hugging Face.


```python
from torch.utils.data import DataLoader
import opencood.hypes_yaml.yaml_utils as yaml_utils
from opencood.tools import train_utils
from opencood.data_utils.datasets import build_dataset
from opencood.utils.seg_utils import (
    cal_iou_training,
    cal_ece_brier_score,
    cal_nll_brier_score
)

dataset = build_dataset(hypes, visualize=False, train=False)

    loader = DataLoader(
        dataset,
        batch_size=1,
        shuffle=False,
        num_workers=args.num_workers,
        collate_fn=dataset.collate_batch,
        pin_memory=False,
        drop_last=False
    )

    print("Loading model...")
    model = train_utils.create_model(hypes)

    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model.to(device)
    print(model)

    _, model = train_utils.load_saved_model(args.model_dir, model)
    model.eval()

    # ---------------- RICH PROGRESS ----------------
    with torch.no_grad():
        with Live(refresh_per_second=4, console=console) as live:
            with Progress(
                TextColumn("Inference"),
                BarColumn(),
                TextColumn("{task.completed}/{task.total}"),
                TimeElapsedColumn(),
                TimeRemainingColumn(),
            ) as progress:

                task = progress.add_task("run", total=total)

                for i, batch_data in enumerate(loader):

                    batch_data = train_utils.to_device(batch_data, device)

                    model_out = model(batch_data['ego'])
                    post_output = dataset.post_process(batch_data['ego'], model_out)

                    # Segmentation Metric
                    iou_d, iou_s = cal_iou_training(batch_data, post_output)

                    ##### Uncertainty Metrics #########
                    ece, ece_eqp, _ = cal_ece_brier_score(batch_data, post_output)
                    nll, brier = cal_nll_brier_score(batch_data, post_output)

  
```

## Citation

```bibtex
@article{jagtap2026hyper,
  title={Hyper-V2X: Hypernetworks for Estimating Epistemic and Aleatoric Uncertainty in Cooperative Bird's-Eye-View Semantic Segmentation},
  author={Jagtap, Abhishek Dinkar and Sadashivaiah, Sanath Tiptur and Festag, Andreas},
  journal={arXiv preprint arXiv:2605.21309},
  year={2026}
}
```