File size: 2,836 Bytes
85b84c2 7fd66d3 85b84c2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
license: apache-2.0
language:
- en
- zh
library_name: transformers
base_model: Qwen/Qwen3-Omni-30B-A3B-Captioner
tags:
- audio
- audio-language-model
- instruction-following
- rubric-based-evaluation
- judge-model
pipeline_tag: audio-text-to-text
---
# AnyAudio-Judge-30B
`AnyAudio-Judge-30B` is the **dynamic rubric-based audio judge** reported in the paper. It is initialized from [Qwen3-Omni-30B-A3B-Captioner](https://huggingface.co/Qwen) and fine-tuned on the AnyAudio-Judge SFT Corpus.
For each binary rubric item describing one verifiable aspect of an audio caption, the model predicts yes / no and produces a one-sentence evidence string drawn from what it heard in the audio. Aggregating the per-item soft probabilities yields a fine-grained alignment score that is significantly more sensitive to partial mismatches than a single holistic match/mismatch judgment.
> Companion benchmark: [`cucl2/AnyAudio-Judge-Bench`](https://huggingface.co/datasets/cucl2/AnyAudio-Judge-Bench)
> Companion corpus: [`cucl2/AnyAudio-Judge-Corpus`](https://huggingface.co/datasets/cucl2/AnyAudio-Judge-Corpus)
> Smaller variant: [`cucl2/AnyAudio-Judge-7B`](https://huggingface.co/cucl2/AnyAudio-Judge-7B)
## Headline numbers (AnyAudio-Judge Bench, accuracy ↑)
| Model | Avg (en) | Avg (zh) |
|---|---:|---:|
| Qwen3-Omni-30B-A3B-Captioner (dynamic rubric, no fine-tuning) | 76.77 | 76.66 |
| Gemini-2.5-Pro (holistic) | 77.72 | 80.01 |
| **AnyAudio-Judge-30B (this checkpoint)** | **84.45** | **85.26** |
## Training
- **Base**: Qwen3-Omni-30B-A3B-Captioner
- **Corpus**: 105K (audio, instruction, rubric, CoT) tuples
- **Stage 1 — SFT**: full-parameter fine-tuning, 1 epoch, 16 × H20 96GB, lr 1e-5, per-device bs 4
- **Stage 2 — GRPO** (separate release): LoRA r=16, α=32, 1 epoch on 8,454 hard samples
This release contains the **SFT-only** stage (matching the "+SFT" row of the ablation table). The +GRPO improvement reported in the paper can be reproduced by running GRPO on top of this checkpoint.
## Usage
```python
from anyaudio_judge import AnyAudioJudge, decompose_instruction
caption = "A gentle, delicate female voice, with soft and smooth pitch, calm and restrained throughout."
rubric = decompose_instruction(caption)
judge = AnyAudioJudge.from_pretrained("cucl2/AnyAudio-Judge-30B")
result = judge.judge("./demo.wav", rubric)
print("alignment_score:", result.score)
for item in result.items:
print(item.question, "->", item.answer)
```
## License
Apache-2.0, inheriting the license of the base Qwen3-Omni-30B-A3B-Captioner model.
## Citation
```bibtex
@misc{anyaudiojudge2026,
title = {AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following},
author = {Anonymous Authors},
year = {2026},
note = {Preprint, under submission}
}
```
|