Streamalign (R16)
Complete R16 speech tokenizer stack. Paired SLM: Streamalign-SLM-R16
(reported metrics: SALMon 69.1, StoryCloze 72.1).
Contents & how the tokenizer uses them
| Path | Role |
|---|---|
rvq_teacher/epoch_16.pt |
The acoustic tokenizer. Contains the fine-tuned streaming encoder and the R16 residual-VQ quantizer (hubert_state_dict: 408 encoder keys + residual_vq). This is the checkpoint that actually produces the R16 speech units. R16-specific. |
alignment_model/ |
Char-level streaming Conformer-Transducer (CKPT+2026-03-15, alignment.yaml). Used as the base model the tokenizer is constructed from; supplies the RNN-T predictor/joiner for char-level alignment. Its encoder is overridden at load by rvq_teacher. R-independent. |
streaming_asr/ |
Word-level streaming Conformer-Transducer (3419_fastemit, chunk_streaming_word_fastemit.yaml, tokenizer.ckpt). Generates the chunk TextGrids (word alignment) and drives the boundary classifier. R-independent. |
boundary_classifier/ |
Word-boundary detector for streaming chunking; consumes streaming_asr. R-independent. |
alignment.yaml |
hparams for the extractor / alignment_model. |
Pipeline
audio -> streaming_asr (word chunks + TextGrids) + boundary_classifier
-> alignment_model scaffold + rvq_teacher encoder -> R16 RVQ units.
Only rvq_teacher/epoch_16.pt is R16-specific; the ASR / alignment /
boundary components are shared across R8/R16/R32.