README.md

---
license: apache-2.0
library_name: pytorch
tags:
- audio
- speech
- frozen-llm
- prealignslm
---

# Audio-align 1.5B Branch Checkpoint

This repository stores the non-main-LLM branches for the PreAlignSLM 1.5B
checkpoint `phase2_3_chat_step5000`. The Qwen2.5-1.5B-Instruct backbone was frozen
throughout training and is intentionally not included.

## Files

| File | Contents |
| --- | --- |
| `semantic_branch.safetensors` | SenseVoice semantic branch, CTC layer, TASU projector, semantic projector |
| `acoustic_branch.safetensors` | ArCap acoustic branch and acoustic recognition projector |
| `router.safetensors` | BERT-small instruction router |
| `manifest.json` | Export metadata, tensor counts, SHA256 checksums |
| `config/` | Training and base config snapshots |

Total exported parameters: `1,795,187,472`.

## Loading

Instantiate the matching PreAlignSLM architecture from this codebase, load the
frozen Qwen2.5-1.5B-Instruct backbone separately, then merge the branch files:

```python
from safetensors.torch import load_file

state = {}
for filename in [
    "semantic_branch.safetensors",
    "acoustic_branch.safetensors",
    "router.safetensors",
]:
    state.update(load_file(filename, device="cpu"))

missing, unexpected = model.load_state_dict(state, strict=False)
assert not unexpected
```

`module.llm.*` keys are expected to be missing from these files because the
main LLM is loaded from its original Qwen checkpoint.

## Reported Metrics

Best open-ended 1.5B checkpoint (`phase2_3_chat/model-5000.pt`):

| Metric | Result |
| --- | --- |
| ASR test-clean / test-other / combined WER | 3.50% / 7.22% / 5.36% |
| AIR-Bench Chat overall, Qwen3.5-27B judge | 2.365 / 10 |
| VoiceBench direct alpacaeval / commoneval / wildvoice | 1.62 / 1.61 / 1.33 |
| VoiceBench direct sd-qa | 21.8% |
| VoiceBench frozen-LLM agent alpacaeval / commoneval / wildvoice | 2.414 / 2.346 / 1.978 |
| VoiceBench frozen-LLM agent sd-qa | 28.9% |