Upload VLA tokenizer with model card

README.md

@@ -10,18 +10,24 @@ tags:
 license: apache-2.0
 ---
 
-# VLA Tokenizer — Adaptive v2 (GPT-NeoX-20b + SNAC)
+# VLA Tokenizer — Adaptive v2 (GPT-NeoX-20b + SNAC + caption/speech)
 
 Extended GPT-NeoX-20b tokenizer for the **FineVideo-VLA** multimodal dataset.
-Adds 3D human pose tokens, video tokens, and SNAC audio tokens on top of the
+Adds 3D human pose tokens, video tokens, SNAC audio tokens, and caption/speech
+wrapper tokens on top of the
 [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) base.
 
-**Vocab size: 156,505** (50,277 base + 93,938 VLA + 12,290 SNAC)
+**Vocab size: 156,509** (50,277 base + 93,938 VLA + 12,290 SNAC + 4 caption/speech)
 
 > **v1 → v2 change:** Added 12,290 SNAC audio tokens (`<snac>`, `</snac>`,
 > and 12,288 `<snac_N>` tokens) for the SNAC listen format used in
 > [MixtureVitae-Omni](https://huggingface.co/datasets/mixture-vitae/MixtureVitae-Omni)
 > and FineVideo-VLA audio tokenization. All existing v1 token IDs are unchanged.
+>
+> **Later addition (same v2 repo, in place):** Added 4 wrapper tokens —
+> `<caption>`, `</caption>`, `<speech>`, `</speech>` — for inline visual
+> caption / spoken-dialogue interleaving at modality-transition points in the
+> token sequence. All prior token IDs (including SNAC) are unchanged.
 
 ---
 
@@ -41,8 +47,9 @@ Adds 3D human pose tokens, video tokens, and SNAC audio tokens on top of the
 | **SNAC Level 1 even** | `<snac_132362>` – `<snac_136457>` | 4,096 | 25 Hz fine audio (even frames) |
 | **SNAC Level 1 odd** | `<snac_144650>` – `<snac_148745>` | 4,096 | 25 Hz fine audio (odd frames) |
 | **SNAC wrappers** | `<snac>`, `</snac>` | 2 | Block delimiters |
+| **Caption/speech wrappers** | `<caption>`, `</caption>`, `<speech>`, `</speech>` | 4 | Inline caption/dialogue interleaving |
 
-**Total new tokens: 106,228** (93,938 VLA + 12,290 SNAC)
+**Total new tokens: 106,232** (93,938 VLA + 12,290 SNAC + 4 caption/speech)
 
 ---
 
@@ -84,7 +91,7 @@ SNAC listen format: 3 tokens per base frame (L0 + L1_even + L1_odd),
 from transformers import AutoTokenizer
 
 tok = AutoTokenizer.from_pretrained("EmpathicRobotics/tokenizer-vla-adaptive-v2")
-print(len(tok))  # 156505
+print(len(tok))  # 156509
 
 # All VLA and SNAC tokens are single atomic tokens
 print(tok.encode("<seed2_1137>",   add_special_tokens=False))  # [59908]
@@ -115,7 +122,7 @@ tok.save_pretrained("tokenizer-vla-adaptive-v2")
 # vocab size: 156,505
 ```
 
-Script: `tools/build_tokenizers.py` in the
+Script: `tools/tokenizer/build_tokenizers.py` in the
 [finevideo-vla](https://github.com/TieuDaoChanNhan/finevideo-vla) repo.
 
 ---

tokenizer.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9d3355c4ef69a8c0a222f2b4bbdccedc4d309454ca9f467c401840f2b35f42c7
-size 23746914
+oid sha256:464e1f6ca5869a2a480bc799df23823b405c579972a73e5063a3d43112f6c43c
+size 23747658