README.md

---
license: apache-2.0
language:
  - en
  - zh
library_name: transformers
base_model: Qwen/Qwen3-Omni-30B-A3B-Captioner
tags:
  - audio
  - audio-language-model
  - instruction-following
  - rubric-based-evaluation
  - judge-model
pipeline_tag: audio-text-to-text
---

# AnyAudio-Judge-30B

`AnyAudio-Judge-30B` is the **dynamic rubric-based audio judge** reported in the paper. It is initialized from [Qwen3-Omni-30B-A3B-Captioner](https://huggingface.co/Qwen) and fine-tuned on the AnyAudio-Judge SFT Corpus.

For each binary rubric item describing one verifiable aspect of an audio caption, the model predicts yes / no and produces a one-sentence evidence string drawn from what it heard in the audio. Aggregating the per-item soft probabilities yields a fine-grained alignment score that is significantly more sensitive to partial mismatches than a single holistic match/mismatch judgment.

> Companion benchmark: [`cucl2/AnyAudio-Judge-Bench`](https://huggingface.co/datasets/cucl2/AnyAudio-Judge-Bench)
> Companion corpus: [`cucl2/AnyAudio-Judge-Corpus`](https://huggingface.co/datasets/cucl2/AnyAudio-Judge-Corpus)
> Smaller variant: [`cucl2/AnyAudio-Judge-7B`](https://huggingface.co/cucl2/AnyAudio-Judge-7B)

## Headline numbers (AnyAudio-Judge Bench, accuracy ↑)

| Model | Avg (en) | Avg (zh) |
|---|---:|---:|
| Qwen3-Omni-30B-A3B-Captioner (dynamic rubric, no fine-tuning) | 76.77 | 76.66 |
| Gemini-2.5-Pro (holistic) | 77.72 | 80.01 |
| **AnyAudio-Judge-30B (this checkpoint)** | **84.45** | **85.26** |

## Training

- **Base**: Qwen3-Omni-30B-A3B-Captioner
- **Corpus**: 105K (audio, instruction, rubric, CoT) tuples
- **Stage 1 — SFT**: full-parameter fine-tuning, 1 epoch, 16 × H20 96GB, lr 1e-5, per-device bs 4
- **Stage 2 — GRPO** (separate release): LoRA r=16, α=32, 1 epoch on 8,454 hard samples

This release contains the **SFT-only** stage (matching the "+SFT" row of the ablation table). The +GRPO improvement reported in the paper can be reproduced by running GRPO on top of this checkpoint.

## Usage

```python
from anyaudio_judge import AnyAudioJudge, decompose_instruction

caption = "A gentle, delicate female voice, with soft and smooth pitch, calm and restrained throughout."
rubric  = decompose_instruction(caption)

judge = AnyAudioJudge.from_pretrained("cucl2/AnyAudio-Judge-30B")
result = judge.judge("./demo.wav", rubric)
print("alignment_score:", result.score)
for item in result.items:
    print(item.question, "->", item.answer)
```

## License

Apache-2.0, inheriting the license of the base Qwen3-Omni-30B-A3B-Captioner model.

## Citation

```bibtex
@misc{anyaudiojudge2026,
  title  = {AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following},
  author = {Anonymous Authors},
  year   = {2026},
  note   = {Preprint, under submission}
}
```