---
license: apache-2.0
base_model:
- Qwen/Qwen2.5-7B-Instruct
- qqjz/WhisperPro-30Plus
library_name: pytorch
pipeline_tag: audio-text-to-text
language:
- en
tags:
- textpro-slm
- prosody
- speech
- audio
- long-audio
---
# TextPro-SLM-7B-30Plus

**Your speech LLM can be a prosody-aware text LLM.**

TextPro-SLM combines transcript tokens with a compact prosody embedding. Transcript tokens carry linguistic content, while prosody carries complementary acoustic information. This input-side factorization brings spoken input closer to the native input format of a text LLM.

[Paper](https://openreview.net/forum?id=Jsjmo8eX29) · [GitHub project](https://github.com/dreamtheater123/TextPro-SLM) · [Paired checkpoint](https://huggingface.co/qqjz/WhisperPro-30Plus)

## Release and pairing

This repository contains the **30Plus extension** and is paired with [WhisperPro-30Plus](https://huggingface.co/qqjz/WhisperPro-30Plus).

The paired inference design handles recordings longer than 30 seconds through VAD segmentation, previous-transcript context, transcript concatenation, and equal-weight averaging of segment prosody vectors. Each Whisper segment remains at most 30 seconds; this is not a native extension of Whisper's context window. This release is separate from the original checkpoints, and original-paper scores are not measurements of the 30Plus extension.

| Component | Specification |
| --- | --- |
| Text backbone | Qwen2.5-7B-Instruct, fully fine-tuned |
| Prosody projector | 1280 → 3584 → 3584, GELU, BF16 |
| Fusion | One global prosody embedding prepended to transcript tokens |
| Audio preprocessing | Mono, 16 kHz |
| LLM context | 32,768 tokens |
| Output | Text response |

## Method

The accompanying paper studies how reducing the speech–text mismatch from the input side can preserve semantic capabilities while learning paralinguistic understanding, using roughly **1,000 hours of LLM training audio** for the paper models. This model card documents the released artifact; it does not introduce new benchmark measurements.

## Download

```bash
pip install "huggingface-hub>=0.36,<1"
hf download qqjz/TextPro-SLM-7B-30Plus --local-dir TextPro-SLM-7B-30Plus
```

This Hub repository contains **checkpoint artifacts and model documentation only**. Inference code, dependencies, and usage examples are being prepared separately in the [GitHub project](https://github.com/dreamtheater123/TextPro-SLM); they are not included in this checkpoint release. Prosody-aware inference requires that implementation rather than loading the text backbone alone.

## Files and export

```text
llm/                     Fine-tuned LLM weights, configuration, and tokenizer
projector.safetensors    Compact trained prosody projector
textpro_config.json      Encoder pairing, fusion, and prosody-token settings
SHA256SUMS.json          Artifact sizes and SHA-256 hashes
```

The supplied tokenizer preserves the trained `<|repo_name|>` prosody marker (ID 151663). The configuration pins the paired encoder revision. The projector is stored as compact safetensors rather than a serialized training storage buffer.

## Model family

| Encoder | Paired 7B model | Audio handling |
| --- | --- | --- |
| [WhisperPro](https://huggingface.co/qqjz/WhisperPro) | [TextPro-SLM-7B-Global](https://huggingface.co/qqjz/TextPro-SLM-7B-Global) | Original, at most 30 seconds |
| [WhisperPro-30Plus](https://huggingface.co/qqjz/WhisperPro-30Plus) | [TextPro-SLM-7B-30Plus](https://huggingface.co/qqjz/TextPro-SLM-7B-30Plus) | Segmentation-based long-audio extension |

## Limitations

Responses may contain transcription errors or hallucinations. Performance can vary with language, recording conditions, speaker characteristics, and domain. Global pooling compresses time-varying prosody. The model returns text; it does not synthesize speech. The 30Plus processing design is subject to segmentation effects and the LLM's finite transcript context; no unlimited-duration or long-audio accuracy claim is implied.

## License and acknowledgements

Apache-2.0; see `LICENSE` and `NOTICE`. The LLM is derived from [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct). The paired WhisperPro encoder is derived from OpenAI Whisper large-v3.

## Citation

```bibtex
@inproceedings{cui2026textproslm,
  title={Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLM},
  author={Cui, Wenqian and Li, Xiao-Hui and Tan, Daxin and Zheng, Qiyong and King, Irwin},
  booktitle={Advances in Neural Information Processing Systems},
  year={2026},
  url={https://openreview.net/forum?id=Jsjmo8eX29}
}
```