File size: 1,954 Bytes
2699c7b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
license: apache-2.0
library_name: pytorch
tags:
- audio
- speech
- frozen-llm
- prealignslm
---
# Audio-align 1.5B Branch Checkpoint
This repository stores the non-main-LLM branches for the PreAlignSLM 1.5B
checkpoint `phase2_3_chat_step5000`. The Qwen2.5-1.5B-Instruct backbone was frozen
throughout training and is intentionally not included.
## Files
| File | Contents |
| --- | --- |
| `semantic_branch.safetensors` | SenseVoice semantic branch, CTC layer, TASU projector, semantic projector |
| `acoustic_branch.safetensors` | ArCap acoustic branch and acoustic recognition projector |
| `router.safetensors` | BERT-small instruction router |
| `manifest.json` | Export metadata, tensor counts, SHA256 checksums |
| `config/` | Training and base config snapshots |
Total exported parameters: `1,795,187,472`.
## Loading
Instantiate the matching PreAlignSLM architecture from this codebase, load the
frozen Qwen2.5-1.5B-Instruct backbone separately, then merge the branch files:
```python
from safetensors.torch import load_file
state = {}
for filename in [
"semantic_branch.safetensors",
"acoustic_branch.safetensors",
"router.safetensors",
]:
state.update(load_file(filename, device="cpu"))
missing, unexpected = model.load_state_dict(state, strict=False)
assert not unexpected
```
`module.llm.*` keys are expected to be missing from these files because the
main LLM is loaded from its original Qwen checkpoint.
## Reported Metrics
Best open-ended 1.5B checkpoint (`phase2_3_chat/model-5000.pt`):
| Metric | Result |
| --- | --- |
| ASR test-clean / test-other / combined WER | 3.50% / 7.22% / 5.36% |
| AIR-Bench Chat overall, Qwen3.5-27B judge | 2.365 / 10 |
| VoiceBench direct alpacaeval / commoneval / wildvoice | 1.62 / 1.61 / 1.33 |
| VoiceBench direct sd-qa | 21.8% |
| VoiceBench frozen-LLM agent alpacaeval / commoneval / wildvoice | 2.414 / 2.346 / 1.978 |
| VoiceBench frozen-LLM agent sd-qa | 28.9% |
|