AnyAudio-Judge-7B

AnyAudio-Judge-7B is a dynamic rubric-based audio judge built on top of Qwen2.5-Omni-7B. It predicts, for each yes/no rubric item describing one verifiable aspect of an audio caption, whether the audio satisfies that aspect — together with a short evidence string.

This is the smaller variant of the AnyAudio-Judge family. The larger AnyAudio-Judge-30B (initialized from Qwen3-Omni-30B-A3B-Captioner) is the variant reported in the paper. The 7B model is trained on the same SFT corpus and is intended for users who need a more efficient evaluator.

Companion benchmark: cucl2/AnyAudio-Judge-Bench Companion corpus: cucl2/AnyAudio-Judge-Corpus Companion 30B model: cucl2/AnyAudio-Judge-30B

Training

• Base: Qwen2.5-Omni-7B
• Corpus: 105K (audio, instruction, rubric, CoT) tuples (see cucl2/AnyAudio-Judge-Corpus)
• Stage: full-parameter SFT for 1 epoch
  • 16 × H20 96GB
  • per-device batch size 4, grad accumulation 1
  • learning rate 1e-5

Usage

from anyaudio_judge import AnyAudioJudge, decompose_instruction

caption = "A gentle, delicate female voice, with soft and smooth pitch, calm and restrained throughout."
rubric  = decompose_instruction(caption)  # external LLM call

judge = AnyAudioJudge.from_pretrained("cucl2/AnyAudio-Judge-7B")
result = judge.judge("./demo.wav", rubric)
print("alignment_score:", result.score)
for item in result.items:
    print(item.question, "->", item.answer)

(See the GitHub repo for the full pipeline including external rubric decomposition.)

License

Apache-2.0, inheriting the license of the base Qwen2.5-Omni-7B model.

Citation

@misc{anyaudiojudge2026,
  title  = {AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following},
  author = {Anonymous Authors},
  year   = {2026},
  note   = {Preprint, under submission}
}