---
title: AVGen-Bench Leaderboard
sdk: gradio
app_file: app.py
pinned: false
license: mit
---

# AVGen-Bench Leaderboard

This is a Hugging Face Space for the AVGen-Bench Text-to-Audio-Video generation leaderboard.

The app loads scores from `data/leaderboard.csv`, renders an interactive leaderboard, and documents the Scheme 2 aggregate metric used by AVGen-Bench.

## Local Run

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python app.py
```

## Deploy to Hugging Face Spaces

```bash
git init
git branch -M main
git add .
git commit -m "Initial AVGen-Bench leaderboard Space"

hf auth login
hf repos create spaces/<your-username>/AVGen-Bench-Leaderboard --type space --space-sdk gradio
git remote add space https://huggingface.co/spaces/<your-username>/AVGen-Bench-Leaderboard
git push space main
```

## Updating Results

Edit `data/leaderboard.csv` with one row per model. The expected columns are:

```text
Model, Components, Component Type, Vis, Aud (PQ), AV, Lip, Text, Face, Music,
Speech, Lo-Phy, Hi-Phy, Holistic, Total, Source
```

`data/submission_template.csv` provides a one-row template for new submissions.

## Public Submissions

The Space includes a `Submission` tab and a Gradio API endpoint named `submit_score`.
Submitted entries are treated as pending review:

1. Users submit raw metric values, model metadata, a public contact, and evaluation artifact links.
2. The app recomputes `Total` from the raw metrics using the AVGen-Bench Scheme 2 formula.
3. The submission is written to a review backend; accepted entries should then be merged into `data/leaderboard.csv`.

Production backend:

```bash
SUBMISSION_BACKEND=github_issue
GITHUB_REPO=<owner>/<repo>
GITHUB_TOKEN=<token-with-issues-write-access>
```

Store `GITHUB_TOKEN` as a Hugging Face Space secret, not in source control. If those variables are absent, the app falls back to `local_file` and writes JSON packets under `pending_submissions/`; for a production Space, use the GitHub issue backend or set `PENDING_SUBMISSION_DIR=/data/pending_submissions` with persistent Space storage.

## Sources

Leaderboard values and the overview figure were initialized from `microsoft/AVGen-Bench` commit `1049eab`.

- Project: https://github.com/microsoft/AVGen-Bench
- Paper: https://arxiv.org/abs/2604.08540
- Dataset: https://huggingface.co/datasets/microsoft/AVGen-Bench