README.md

metadata

title: AVGen-Bench Leaderboard
sdk: gradio
app_file: app.py
pinned: false
license: mit

AVGen-Bench Leaderboard

This is a Hugging Face Space for the AVGen-Bench Text-to-Audio-Video generation leaderboard.

The app loads scores from data/leaderboard.csv, renders an interactive leaderboard, and documents the Scheme 2 aggregate metric used by AVGen-Bench.

Local Run

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python app.py

Deploy to Hugging Face Spaces

git init
git branch -M main
git add .
git commit -m "Initial AVGen-Bench leaderboard Space"

hf auth login
hf repos create spaces/<your-username>/AVGen-Bench-Leaderboard --type space --space-sdk gradio
git remote add space https://huggingface.co/spaces/<your-username>/AVGen-Bench-Leaderboard
git push space main

Updating Results

Edit data/leaderboard.csv with one row per model. The expected columns are:

Model, Components, Component Type, Vis, Aud (PQ), AV, Lip, Text, Face, Music,
Speech, Lo-Phy, Hi-Phy, Holistic, Total, Source

data/submission_template.csv provides a one-row template for new submissions.

Public Submissions

The Space includes a Submission tab and a Gradio API endpoint named submit_score. Submitted entries are treated as pending review:

1. Users submit raw metric values, model metadata, a public contact, and evaluation artifact links.
2. The app recomputes Total from the raw metrics using the AVGen-Bench Scheme 2 formula.
3. The submission is written to a review backend; accepted entries should then be merged into data/leaderboard.csv.

Production backend:

SUBMISSION_BACKEND=github_issue
GITHUB_REPO=<owner>/<repo>
GITHUB_TOKEN=<token-with-issues-write-access>

Store GITHUB_TOKEN as a Hugging Face Space secret, not in source control. If those variables are absent, the app falls back to local_file and writes JSON packets under pending_submissions/; for a production Space, use the GitHub issue backend or set PENDING_SUBMISSION_DIR=/data/pending_submissions with persistent Space storage.

Sources

Leaderboard values and the overview figure were initialized from microsoft/AVGen-Bench commit 1049eab.

• Project: https://github.com/microsoft/AVGen-Bench
• Paper: https://arxiv.org/abs/2604.08540
• Dataset: https://huggingface.co/datasets/microsoft/AVGen-Bench