Streamalign (R16)
Complete R16 speech tokenizer stack. Paired SLM: Streamalign-SLM-R16
(reported metrics: SALMon 69.1, StoryCloze 72.1).
Contents & roles
| Path | Role |
|---|---|
rvq_teacher/ |
The acoustic tokenizer: the streaming encoder plus the R16 residual-VQ quantizer. Produces the R16 speech units. R16-specific. |
alignment_model/ |
Char-level streaming Conformer-Transducer. Base scaffold the tokenizer is built on; supplies the RNN-T predictor/joiner for char-level alignment. R-independent. |
streaming_asr/ |
Word-level streaming Conformer-Transducer. Generates the chunk TextGrids (word alignment) and drives the boundary classifier. R-independent. |
boundary_classifier/ |
Word-boundary detector for streaming chunking; consumes streaming_asr. R-independent. |
alignment.yaml |
Extractor / alignment_model hparams. |
Pipeline
audio -> streaming_asr (word chunks + TextGrids) + boundary_classifier
-> alignment_model scaffold + rvq_teacher encoder -> R16 RVQ units.
Only rvq_teacher is R16-specific; the ASR / alignment / boundary
components are shared across R8/R16/R32.
Tokenizer checkpoint
rvq_teacher/epoch_22.pt is the final R16 tokenizer, trained on LibriSpeech
plus the full Emilia set with a learnable codebook at a 10x codebook learning
rate (260512_RVQ_R16_C512_nodistill_codelr10x_LS_emilia_full_from_nofsq_e13).
It replaces the earlier LibriSpeech-only epoch_16.pt.
Load it with RVQ_R=16 and RVQ_CODEBOOK_SIZE=512, and leave
RVQ_CODEBOOK_DIM unset: codebook_dim defaults to feat_dim=256, which is
what these weights expect.