README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: feature-extraction
base_model: Qwen/Qwen2.5-Omni-7B
tags:
  - speech
  - audio-encoder
  - flexislm
  - qwen2.5-omni
---

# Qwen2.5-Omni Audio Encoder

Standalone audio encoder extracted from [Qwen/Qwen2.5-Omni-7B](https://huggingface.co/Qwen/Qwen2.5-Omni-7B) for [FlexiSLM](https://huggingface.co/FlexiSLM).

This checkpoint keeps only `Qwen2_5OmniAudioEncoder` (`thinker.audio_tower.*`: conv layers, 32 transformer layers, `ln_post`, and `proj`). The Omni thinker LLM, talker, vision tower, and tokenizer are omitted.

| Property | Value |
| --- | --- |
| Parameters | 640M |
| Size | ~1.19 GiB (`bfloat16`) |
| Output dim | 3584 |
| Mel bins | 128 |

- Paper: [arXiv:2606.31247](https://arxiv.org/abs/2606.31247)
- FlexiSLM code: [AmphionTeam/FlexiSLM](https://github.com/AmphionTeam/FlexiSLM)

## Usage with FlexiSLM

Required for both 7B and 0.5B FlexiSLM inference and training. Shared by all Stage 1 / Stage 2 / Stage 3 recipes.

### Download

```bash
MODEL_ROOT="$PWD/models"
hf download FlexiSLM/Qwen2_5-Omni-Audio_Encoder \
  --local-dir "$MODEL_ROOT/Qwen2_5-Omni-Audio_Encoder"
```

With FlexiSLM `auto_download=True`, this encoder is fetched automatically together with the Stage 2 SLM, SenseVoice, and FlexiCodec assets.

### Inference config (Python)

```python
from pathlib import Path
from src.inference_flexislm import FlexiSLMInferenceConfig

model_root = Path.cwd() / "models"
config = FlexiSLMInferenceConfig(
    checkpoint="stage2_7B",  # or "stage2_0.5B"
    model_path=str(model_root / "FlexiSLM-7B-Stage2"),
    qwen25o_encoder_path=str(model_root / "Qwen2_5-Omni-Audio_Encoder"),
    qwen25o_encoder_config_path=str(
        model_root / "Qwen2_5-Omni-Audio_Encoder/config.json"
    ),
    # ... FlexiCodec / SenseVoice / flow-matching paths ...
)
```

### Training / YAML

```yaml
use_qwen25omni_feature: true
qwen25omni_encoder_path: models/Qwen2_5-Omni-Audio_Encoder
qwen25omni_encoder_config_path: models/Qwen2_5-Omni-Audio_Encoder/config.json
```

Full TTS / ASR / audio QA / speech-to-speech examples: [FlexiSLM Inference Guide](https://github.com/AmphionTeam/FlexiSLM#inference).

## Load with Transformers

Weights are loadable with `transformers.models.qwen2_5_omni.modeling_qwen2_5_omni.Qwen2_5OmniAudioEncoder`.

## License

Apache 2.0, same as the source Qwen2.5-Omni-7B checkpoint. Copyright 2025 Alibaba Cloud.

## Citation

If you use this encoder as part of FlexiSLM:

```bibtex
@misc{li2026flexislmdynamiccontrollableframe,
      title={FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model},
      author={Jiaqi Li and Chaoren Wang and Xiaohai Tian and Mingjie Chen and Xinyu Liang and Xu Li and Yufan Lin and Junwen Qiu and Jun Zhang and Lu Lu and Haizhou Li and Zhizheng Wu},
      year={2026},
      eprint={2606.31247},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2606.31247},
}
```