| title: NAVA Audio-Video Generator | |
| emoji: 🎬 | |
| colorFrom: purple | |
| colorTo: blue | |
| sdk: gradio | |
| sdk_version: 6.16.0 | |
| app_file: app.py | |
| models: | |
| - ernie-research/NAVA | |
| - Qwen/Qwen3-4B-Instruct-2507 | |
| - Qwen/Qwen3-VL-4B-Instruct | |
| short_description: Native AV alignment — joint video + audio generation | |
| license: apache-2.0 | |
| # NAVA on ZeroGPU | |
| [Project Page](https://ernie-research.github.io/NAVA) · [arXiv](https://arxiv.org/abs/2605.30073) · [GitHub](https://github.com/ernie-research/NAVA) · [Models](https://huggingface.co/ernie-research/NAVA) | |
| NAVA is a Native Audio-Visual Alignment framework for **joint** audio + video generation: | |
| text → synchronized 24fps video + stereo audio in one shot, image-to-AV with reference-voice timbre control included. This Space runs the FP8 checkpoint (~13 GB) on a single H200 via ZeroGPU. | |
| ## How it works | |
| 1. **(Optional) Upload a first-frame image** — the aspect ratio dropdown auto-detects portrait/landscape/square. | |
| 2. **Type a short prompt** — Chinese or English, brief is fine. | |
| 3. **Click "Rewrite Prompt"** — Qwen3-VL captions the image (if any), composes with your prompt, and Qwen3 rewrites it into the long, dense Chinese caption NAVA was trained on. Auto-retries on `<S>...<E>` mismatch. | |
| 4. **Click "Generate"** — diffusion runs in FP8 (~5 minutes per 4s clip on H200). | |
| ## Speech tags | |
| Wrap any spoken line in `<S>...<E>`. The rewriter preserves these verbatim (auto-retried until the count matches your input). Add reference WAVs to bind voice identity to each speech span. | |
| ## Hardware | |
| ZeroGPU H200 (80 GB). Each `Generate` call is allocated up to 5 minutes of GPU time. If the queue is busy, you'll wait. The model + Qwen3 weights stay resident in CPU RAM between calls; only the active stage moves to GPU. | |