Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,9 +1,21 @@
|
|
| 1 |
# Streamalign (R16)
|
| 2 |
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
Streamalign-SLM (R16).
|
| 6 |
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# Streamalign (R16)
|
| 2 |
|
| 3 |
+
Complete R16 speech tokenizer stack. Paired SLM: `Streamalign-SLM-R16`
|
| 4 |
+
(reported metrics: SALMon 69.1, StoryCloze 72.1).
|
|
|
|
| 5 |
|
| 6 |
+
## Contents & how the tokenizer uses them
|
| 7 |
+
|
| 8 |
+
| Path | Role |
|
| 9 |
+
|------|------|
|
| 10 |
+
| `rvq_teacher/epoch_16.pt` | **The acoustic tokenizer.** Contains the fine-tuned streaming encoder **and** the R16 residual-VQ quantizer (`hubert_state_dict`: 408 encoder keys + `residual_vq`). This is the checkpoint that actually produces the R16 speech units. R16-specific. |
|
| 11 |
+
| `alignment_model/` | Char-level streaming Conformer-Transducer (`CKPT+2026-03-15`, `alignment.yaml`). Used as the base model the tokenizer is constructed from; supplies the RNN-T predictor/joiner for char-level alignment. Its encoder is overridden at load by `rvq_teacher`. R-independent. |
|
| 12 |
+
| `streaming_asr/` | Word-level streaming Conformer-Transducer (`3419_fastemit`, `chunk_streaming_word_fastemit.yaml`, `tokenizer.ckpt`). Generates the chunk TextGrids (word alignment) and drives the boundary classifier. R-independent. |
|
| 13 |
+
| `boundary_classifier/` | Word-boundary detector for streaming chunking; consumes `streaming_asr`. R-independent. |
|
| 14 |
+
| `alignment.yaml` | hparams for the extractor / `alignment_model`. |
|
| 15 |
+
|
| 16 |
+
## Pipeline
|
| 17 |
+
audio -> `streaming_asr` (word chunks + TextGrids) + `boundary_classifier`
|
| 18 |
+
-> `alignment_model` scaffold + `rvq_teacher` encoder -> R16 RVQ units.
|
| 19 |
+
|
| 20 |
+
Only `rvq_teacher/epoch_16.pt` is R16-specific; the ASR / alignment /
|
| 21 |
+
boundary components are shared across R8/R16/R32.
|