Initial OpenFormosa release of BlueMagpie-TTS (rebrand from voidful; provenance refs preserved)

.gitattributes

@@ -1,35 +1,3 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
 *.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoints/hung_yi_lee_speaker_centroids.pt filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,143 @@
+---
+language:
+- zh
+- en
+tags:
+- text-to-speech
+- speech-synthesis
+- code-switching
+- private
+library_name: bluemagpie
+pipeline_tag: text-to-speech
+license: other
+---
+
+# BlueMagpie-TTS
+
+Private BlueMagpie-TTS checkpoint for internal research and evaluation.
+
+This repository contains the inference artifact only: model weights, AudioVAE
+weights, tokenizer files, config, a default Hung-yi Lee speaker centroid, and
+usage documentation. It does not include optimizer state, scheduler state,
+training logs, local configs, or training-data metadata.
+
+## Intended Use
+
+- Mandarin and mixed Mandarin/English text-to-speech evaluation.
+- Internal experiments with reference-audio prompting, continuation prompting,
+  and optional speaker-centroid conditioning.
+- Do not redistribute the checkpoint or generated speech unless rights and
+  consent are cleared for the intended use.
+
+## Install
+
+```bash
+git clone https://github.com/OpenFormosa/BlueMagpie-TTS
+cd BlueMagpie-TTS
+pip install -e ".[train]"
+pip install soundfile huggingface_hub
+```
+
+## Quick Start
+
+```python
+from huggingface_hub import snapshot_download
+import soundfile as sf
+import torch
+
+from bluemagpie import BlueMagpieModel
+
+model_dir = snapshot_download("OpenFormosa/BlueMagpie-TTS", token=True)
+model = BlueMagpieModel.from_local(model_dir, training=False, device="cuda")
+centroids = torch.load(
+    f"{model_dir}/checkpoints/hung_yi_lee_speaker_centroids.pt",
+    map_location="cpu",
+    weights_only=True,
+)
+speaker_centroid = centroids["centroids"][centroids["speaker_ids"].index("hung_yi_lee")]
+
+audio = model.generate(
+    target_text="這是 AI TTS code switching 測試。",
+    cfg_value=2.8,
+    inference_timesteps=9,
+    max_len=2000,
+    retry_badcase=True,
+    speaker_centroid=speaker_centroid,
+)
+
+sf.write("sample.wav", audio.detach().cpu().numpy(), model.sample_rate)
+```
+
+## Reference Audio Prompting
+
+```python
+audio = model.generate(
+    target_text="今天的會議改到下午三點。",
+    reference_wav_path="reference_speaker.wav",
+    cfg_value=2.8,
+    inference_timesteps=9,
+)
+```
+
+Only use reference audio from speakers you have permission to synthesize.
+
+## Recommended Defaults
+
+The current recommended defaults are also recorded in `config.json` under
+`generation_defaults` and in `release_metadata.json` under
+`recommended_generation_defaults`.
+
+- `cfg_value=2.8`
+- `inference_timesteps=9`
+- `max_len=2000`
+- `retry_badcase=True`
+- default speaker centroid: `checkpoints/hung_yi_lee_speaker_centroids.pt`
+  (`speaker_id="hung_yi_lee"`, source dataset `voidful/hung-yi_lee`)
+
+These defaults were selected on `/home/voidful/tts_hard_sentences_zh_500.txt`
+using `MediaTek-Research/Breeze-ASR-25` with normalized CER. The best trial was
+`hy_cfg2p8_steps9`: CER `0.09669792733863977`, TER `0.0911015155363644`,
+with `1227/12689` character errors.
+
+## Long Text
+
+For long-form synthesis, split text into sentence-sized chunks and concatenate
+the generated waveforms. For stronger continuity, pass a short approved prompt
+clip with `prompt_text` and `prompt_wav_path`, then synthesize the next chunk.
+
+## Evaluation
+
+Numbers below are from an internal held-out evaluation set. The eval set and
+training data are intentionally not described in this private model card.
+
+| System | Setting | CER | WER |
+| --- | --- | ---: | ---: |
+| BlueMagpie-TTS | selected checkpoint | 4.81% | 5.36% |
+| Reference baseline | same internal eval | 11.45% | 14.83% |
+
+Selected checkpoint speed diagnostics on the same internal eval:
+
+| Metric | Value |
+| --- | ---: |
+| Median duration-units/sec | 4.748 |
+| Max duration-units/sec | 5.288 |
+
+## Limitations
+
+- Metrics are not a public benchmark and should be used only for internal model
+  selection.
+- Speaker similarity depends on the quality and rights-cleared status of the
+  supplied reference audio or centroid.
+- Very long passages should be chunked to avoid stop-token and prosody drift.
+- Generated speech may be incorrect; do not use it as a real-world notification
+  without human review.
+
+## Files
+
+- `pytorch_model.bin`: BlueMagpie model weights.
+- `audiovae.pth`: AudioVAE weights.
+- `config.json`: BlueMagpie architecture/runtime config.
+- `tokenizer.json`, `tokenizer_config.json`: tokenizer files.
+- `checkpoints/hung_yi_lee_speaker_centroids.pt`: default speaker centroid
+  table used by the recommended defaults.
+- `USAGE.md`: expanded usage guide.

USAGE.md

@@ -0,0 +1,103 @@
+# BlueMagpie-TTS Usage
+
+This is a private inference checkpoint. Install the local package first:
+
+```bash
+git clone https://github.com/OpenFormosa/BlueMagpie-TTS
+cd BlueMagpie-TTS
+pip install -e ".[train]"
+pip install soundfile huggingface_hub
+```
+
+Download and load:
+
+```python
+from huggingface_hub import snapshot_download
+from bluemagpie import BlueMagpieModel
+
+model_dir = snapshot_download("OpenFormosa/BlueMagpie-TTS", token=True)
+model = BlueMagpieModel.from_local(model_dir, training=False, device="cuda")
+```
+
+Generate a short utterance:
+
+```python
+import soundfile as sf
+import torch
+
+centroids = torch.load(
+    f"{model_dir}/checkpoints/hung_yi_lee_speaker_centroids.pt",
+    map_location="cpu",
+    weights_only=True,
+)
+speaker_centroid = centroids["centroids"][centroids["speaker_ids"].index("hung_yi_lee")]
+
+wav = model.generate(
+    target_text="這是合成語音測試。",
+    cfg_value=2.8,
+    inference_timesteps=9,
+    max_len=2000,
+    retry_badcase=True,
+    speaker_centroid=speaker_centroid,
+)
+sf.write("short.wav", wav.detach().cpu().numpy(), model.sample_rate)
+```
+
+Generate with reference audio:
+
+```python
+wav = model.generate(
+    target_text="明天 meeting 取消一次，我們大家放假。",
+    reference_wav_path="reference_speaker.wav",
+    cfg_value=2.8,
+    inference_timesteps=9,
+)
+```
+
+Streaming:
+
+```python
+import torch
+import soundfile as sf
+
+chunks = []
+for chunk in model.generate_streaming(
+    target_text="這是一段串流語音合成測試。",
+    cfg_value=2.8,
+    inference_timesteps=9,
+):
+    chunks.append(chunk.detach().cpu())
+
+wav = torch.cat(chunks, dim=-1)
+sf.write("streaming.wav", wav.numpy(), model.sample_rate)
+```
+
+Long text:
+
+1. Split the text by sentence or paragraph.
+2. Generate each chunk separately.
+3. Concatenate the waveforms with a short crossfade if needed.
+4. For continuation-style context, pass a short approved prompt clip with
+   `prompt_text` and `prompt_wav_path`.
+
+Recommended generation defaults:
+
+- `cfg_value=2.8`
+- `inference_timesteps=9`
+- `retry_badcase=True` for offline generation
+- `max_len=2000` unless a longer chunk is intentionally needed
+- `speaker_centroid=...` from
+  `checkpoints/hung_yi_lee_speaker_centroids.pt` when using the default
+  `hung_yi_lee` voice
+
+The same defaults are stored in `config.json` as `generation_defaults` and in
+`release_metadata.json` as `recommended_generation_defaults`. They were selected
+from the `hy_cfg2p8_steps9` trial on the 500-sentence hard Chinese TTS set using
+`MediaTek-Research/Breeze-ASR-25`: normalized CER `0.09669792733863977`, TER
+`0.0911015155363644`, `1227/12689` character errors.
+
+Safety:
+
+- Use only rights-cleared reference audio or speaker embeddings.
+- Do not present generated speech as a real person or real notification unless
+  that is explicitly authorized and reviewed.

audiovae.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc01a2d5d817910f185d959f8bb23d1a5ab80c9a03b7398ac86d7457cb663332
+size 376972749

checkpoints/hung_yi_lee_speaker_centroids.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1d4c95a4c33935ff1fee0ab47fa796dcc13908a183c60e9b02bc0a61c541c4c
+size 2691

config.json

@@ -0,0 +1,324 @@
+{
+  "barbet_config": {
+    "attention_dropout": 0.0,
+    "attention_sink": false,
+    "bos_token_id": 114689,
+    "eos_token_id": 114690,
+    "global_attention_layers": [
+      0,
+      4,
+      8,
+      12,
+      16,
+      20,
+      24
+    ],
+    "head_dim": 128,
+    "hidden_dropout": 0.0,
+    "hidden_size": 1536,
+    "initializer_range": 0.02,
+    "intermediate_size": 5120,
+    "mamba_d_conv": 4,
+    "mamba_d_state": 64,
+    "mamba_expand": 2,
+    "mamba_layers": [
+      3,
+      7,
+      11,
+      15,
+      19,
+      23,
+      27
+    ],
+    "max_position_embeddings": 262144,
+    "mtp_enabled": false,
+    "mtp_loss_weights": {
+      "2": 0.2,
+      "3": 0.1
+    },
+    "mtp_offsets": [
+      2,
+      3
+    ],
+    "num_attention_heads": 16,
+    "num_hidden_layers": 28,
+    "num_key_value_heads": 2,
+    "pad_token_id": 114691,
+    "qk_clip_alpha": 0.5,
+    "qk_clip_threshold": 100.0,
+    "qk_logit_clip": false,
+    "qk_norm": true,
+    "rms_norm_eps": 1e-06,
+    "rope_theta": 10000000.0,
+    "sliding_window_size": 8192,
+    "tie_word_embeddings": true,
+    "unk_token_id": 114688,
+    "use_cache": true,
+    "vocab_size": 114944
+  },
+  "vox_lm_config": {
+    "bos_token_id": 1,
+    "eos_token_id": 2,
+    "hidden_size": 2048,
+    "intermediate_size": 6144,
+    "max_position_embeddings": 32768,
+    "num_attention_heads": 16,
+    "num_hidden_layers": 28,
+    "num_key_value_heads": 2,
+    "rms_norm_eps": 1e-05,
+    "rope_scaling": {
+      "type": "longrope",
+      "long_factor": [
+        0.9977997200264581,
+        1.014658295992452,
+        1.0349680404997148,
+        1.059429246056193,
+        1.0888815016813513,
+        1.1243301355211495,
+        1.166977103606075,
+        1.2182568066927284,
+        1.2798772354275727,
+        1.3538666751582975,
+        1.4426259039919596,
+        1.5489853358570191,
+        1.6762658237220625,
+        1.8283407612492941,
+        2.0096956085876183,
+        2.225478927469756,
+        2.481536379650452,
+        2.784415934557119,
+        3.1413289096347365,
+        3.560047844772632,
+        4.048719380066383,
+        4.615569542115128,
+        5.2684819496549835,
+        6.014438591970396,
+        6.858830049237097,
+        7.804668263503327,
+        8.851768731513417,
+        9.99600492938444,
+        11.228766118181639,
+        12.536757560834843,
+        13.902257701387796,
+        15.303885189125953,
+        16.717837610115794,
+        18.119465097853947,
+        19.484965238406907,
+        20.792956681060105,
+        22.02571786985731,
+        23.16995406772833,
+        24.217054535738416,
+        25.16289275000465,
+        26.007284207271347,
+        26.753240849586767,
+        27.40615325712662,
+        27.973003419175363,
+        28.461674954469114,
+        28.880393889607006,
+        29.237306864684626,
+        29.540186419591297,
+        29.79624387177199,
+        30.01202719065413,
+        30.193382037992453,
+        30.34545697551969,
+        30.47273746338473,
+        30.579096895249787,
+        30.66785612408345,
+        30.741845563814174,
+        30.80346599254902,
+        30.85474569563567,
+        30.897392663720595,
+        30.932841297560394,
+        30.962293553185553,
+        30.986754758742034,
+        31.007064503249293,
+        31.02392307921529
+      ],
+      "short_factor": [
+        0.9977997200264581,
+        1.014658295992452,
+        1.0349680404997148,
+        1.059429246056193,
+        1.0888815016813513,
+        1.1243301355211495,
+        1.166977103606075,
+        1.2182568066927284,
+        1.2798772354275727,
+        1.3538666751582975,
+        1.4426259039919596,
+        1.5489853358570191,
+        1.6762658237220625,
+        1.8283407612492941,
+        2.0096956085876183,
+        2.225478927469756,
+        2.481536379650452,
+        2.784415934557119,
+        3.1413289096347365,
+        3.560047844772632,
+        4.048719380066383,
+        4.615569542115128,
+        5.2684819496549835,
+        6.014438591970396,
+        6.858830049237097,
+        7.804668263503327,
+        8.851768731513417,
+        9.99600492938444,
+        11.228766118181639,
+        12.536757560834843,
+        13.902257701387796,
+        15.303885189125953,
+        16.717837610115794,
+        18.119465097853947,
+        19.484965238406907,
+        20.792956681060105,
+        22.02571786985731,
+        23.16995406772833,
+        24.217054535738416,
+        25.16289275000465,
+        26.007284207271347,
+        26.753240849586767,
+        27.40615325712662,
+        27.973003419175363,
+        28.461674954469114,
+        28.880393889607006,
+        29.237306864684626,
+        29.540186419591297,
+        29.79624387177199,
+        30.01202719065413,
+        30.193382037992453,
+        30.34545697551969,
+        30.47273746338473,
+        30.579096895249787,
+        30.66785612408345,
+        30.741845563814174,
+        30.80346599254902,
+        30.85474569563567,
+        30.897392663720595,
+        30.932841297560394,
+        30.962293553185553,
+        30.986754758742034,
+        31.007064503249293,
+        31.02392307921529
+      ],
+      "original_max_position_embeddings": 32768
+    },
+    "vocab_size": 73448,
+    "use_mup": false,
+    "scale_emb": 12.0,
+    "dim_model_base": 256,
+    "scale_depth": 1.4,
+    "rope_theta": 10000.0,
+    "kv_channels": 128,
+    "no_rope": false
+  },
+  "patch_size": 4,
+  "feat_dim": 64,
+  "residual_lm_num_layers": 8,
+  "residual_lm_no_rope": true,
+  "scalar_quantization_latent_dim": 512,
+  "scalar_quantization_scale": 9,
+  "encoder_config": {
+    "hidden_dim": 1024,
+    "ffn_dim": 4096,
+    "num_heads": 16,
+    "num_layers": 12,
+    "kv_channels": 128
+  },
+  "dit_config": {
+    "hidden_dim": 1024,
+    "ffn_dim": 4096,
+    "num_heads": 16,
+    "num_layers": 12,
+    "kv_channels": 128,
+    "dit_mean_mode": false,
+    "cfm_config": {
+      "sigma_min": 1e-06,
+      "solver": "euler",
+      "t_scheduler": "log-norm",
+      "training_cfg_rate": 0.1,
+      "inference_cfg_rate": 2.0,
+      "reg_loss_type": "l1",
+      "ratio_r_neq_t_range": [
+        0.25,
+        0.75
+      ],
+      "noise_cond_prob_range": [
+        0.0,
+        0.0
+      ],
+      "noise_cond_scale": 0.0
+    }
+  },
+  "audio_vae_config": {
+    "encoder_dim": 128,
+    "encoder_rates": [
+      2,
+      5,
+      8,
+      8
+    ],
+    "latent_dim": 64,
+    "decoder_dim": 2048,
+    "decoder_rates": [
+      8,
+      6,
+      5,
+      2,
+      2,
+      2
+    ],
+    "depthwise": true,
+    "sample_rate": 16000,
+    "out_sample_rate": 48000,
+    "use_noise_block": false,
+    "sr_bin_boundaries": [
+      20000,
+      30000,
+      40000
+    ],
+    "cond_type": "scale_bias",
+    "cond_dim": 128,
+    "cond_out_layer": false
+  },
+  "adapter_config": {
+    "num_residual_blocks": 1,
+    "ffn_mult": 2.0,
+    "rms_norm_eps": 1e-06
+  },
+  "speaker_embed_dim": 192,
+  "audio_start_token": -1,
+  "audio_end_token": -1,
+  "ref_audio_start_token": -1,
+  "ref_audio_end_token": -1,
+  "spk_token": -1,
+  "barbet_effective_vocab_size": null,
+  "max_length": 8192,
+  "device": "cuda",
+  "dtype": "bfloat16",
+  "generation_defaults": {
+    "cfg_value": 2.8,
+    "inference_timesteps": 9,
+    "max_len": 2000,
+    "retry_badcase": true,
+    "retry_badcase_max_times": 3,
+    "retry_badcase_ratio_threshold": 6.0,
+    "speaker_id": "hung_yi_lee",
+    "speaker_source_dataset": "voidful/hung-yi_lee",
+    "speaker_centroid_path": "checkpoints/hung_yi_lee_speaker_centroids.pt",
+    "speaker_centroid_sha256": "e1d4c95a4c33935ff1fee0ab47fa796dcc13908a183c60e9b02bc0a61c541c4c",
+    "speaker_centroid_dim": 192
+  },
+  "generation_defaults_source": {
+    "scope": "tts_hard_sentences_zh_500 + Breeze-ASR-25 normalized CER",
+    "sentences": "/home/voidful/tts_hard_sentences_zh_500.txt",
+    "asr_model": "MediaTek-Research/Breeze-ASR-25",
+    "conversion": "s2twp",
+    "normalized_cer": 0.09669792733863977,
+    "mixed_token_error_rate": 0.0911015155363644,
+    "char_errors": 1227,
+    "char_reference_length": 12689,
+    "evaluated_examples": 500,
+    "trial": "hy_cfg2p8_steps9",
+    "run": "hungyi_high_refine_hy_cfg2p8_steps9_20260620"
+  }
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33dc9593a5cf25619853a4140f5114b4fb61954b163c86d28f145321272806dd
+size 7746547131

release_metadata.json

@@ -0,0 +1,45 @@
+{
+  "format": "bluemagpie-pretrained",
+  "checkpoint": "step_0003900",
+  "files": [
+    "pytorch_model.bin",
+    "audiovae.pth",
+    "config.json",
+    "tokenizer.json",
+    "tokenizer_config.json",
+    "checkpoints/hung_yi_lee_speaker_centroids.pt"
+  ],
+  "evaluation": {
+    "scope": "internal held-out evaluation",
+    "cer": 0.048101,
+    "wer": 0.05356,
+    "reference_baseline_cer": 0.114483,
+    "reference_baseline_wer": 0.148266
+  },
+  "recommended_generation_defaults": {
+    "cfg_value": 2.8,
+    "inference_timesteps": 9,
+    "max_len": 2000,
+    "retry_badcase": true,
+    "retry_badcase_max_times": 3,
+    "retry_badcase_ratio_threshold": 6.0,
+    "speaker_id": "hung_yi_lee",
+    "speaker_source_dataset": "voidful/hung-yi_lee",
+    "speaker_centroid_path": "checkpoints/hung_yi_lee_speaker_centroids.pt",
+    "speaker_centroid_sha256": "e1d4c95a4c33935ff1fee0ab47fa796dcc13908a183c60e9b02bc0a61c541c4c",
+    "speaker_centroid_dim": 192
+  },
+  "recommended_generation_defaults_source": {
+    "scope": "tts_hard_sentences_zh_500 + Breeze-ASR-25 normalized CER",
+    "sentences": "/home/voidful/tts_hard_sentences_zh_500.txt",
+    "asr_model": "MediaTek-Research/Breeze-ASR-25",
+    "conversion": "s2twp",
+    "normalized_cer": 0.09669792733863977,
+    "mixed_token_error_rate": 0.0911015155363644,
+    "char_errors": 1227,
+    "char_reference_length": 12689,
+    "evaluated_examples": 500,
+    "trial": "hy_cfg2p8_steps9",
+    "run": "hungyi_high_refine_hy_cfg2p8_steps9_20260620"
+  }
+}

tokenizer_config.json

@@ -0,0 +1,250 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "effective_vocab_size": 114822,
+  "eos_token": "</s>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|bos|>",
+    "<|eos|>",
+    "<|unk|>",
+    "<|pad|>",
+    "<|system|>",
+    "<|user_channel|>",
+    "<|assistant_channel|>",
+    "<|task:speech_to_text|>",
+    "<|task:text_to_speech|>",
+    "<|input_audio_start|>",
+    "<|input_audio_end|>",
+    "<|audio_ref_start|>",
+    "<|audio_ref_end|>",
+    "<|audio_start|>",
+    "<|audio_end|>",
+    "<|speech_start|>",
+    "<|speech_end|>",
+    "<|transcript_start|>",
+    "<|transcript_end|>",
+    "<|segment_start|>",
+    "<|segment_end|>",
+    "<|speaker|>",
+    "<|start_time|>",
+    "<|end_time|>",
+    "<|duration|>",
+    "<|content|>",
+    "<|non_speech_event|>",
+    "<|retrieval_result_start|>",
+    "<|retrieval_result_end|>",
+    "<|ocr_start|>",
+    "<|ocr_end|>",
+    "<|image_start|>",
+    "<|image_end|>",
+    "<|video_start|>",
+    "<|video_end|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|tool_call|>",
+    "<|tool_response|>",
+    "<|endoftext|>",
+    "<think>",
+    "</think>",
+    "<|no_think|>",
+    "<|think|>",
+    "<|think_max|>",
+    "<|task:text_to_text|>",
+    "<|task:speech_to_speech|>",
+    "<|task:text_speech_to_text|>",
+    "<|task:text_speech_to_speech|>",
+    "<|task:full_duplex_speech|>",
+    "<|task:agent|>",
+    "<|task:tool_use|>",
+    "<|task:rag|>",
+    "<|task:code_execution|>",
+    "<|task:document_qa|>",
+    "<|task:data_analysis|>",
+    "<|task:workflow|>",
+    "<|reasoning_mode:none|>",
+    "<|reasoning_mode:short|>",
+    "<|reasoning_mode:deep|>",
+    "<|reasoning_mode:verify|>",
+    "<|private_reasoning_start|>",
+    "<|private_reasoning_end|>",
+    "<|reasoning_summary_start|>",
+    "<|reasoning_summary_end|>",
+    "<|plan_start|>",
+    "<|plan_end|>",
+    "<|step_start|>",
+    "<|step_end|>",
+    "<|action_start|>",
+    "<|action_end|>",
+    "<|observation_start|>",
+    "<|observation_end|>",
+    "<|reflection_start|>",
+    "<|reflection_end|>",
+    "<|verification_start|>",
+    "<|verification_end|>",
+    "<|tool_schema_start|>",
+    "<|tool_schema_end|>",
+    "<|tool_call_start|>",
+    "<|tool_call_end|>",
+    "<|tool_result_start|>",
+    "<|tool_result_end|>",
+    "<|tool_error_start|>",
+    "<|tool_error_end|>",
+    "<|retrieval_query_start|>",
+    "<|retrieval_query_end|>",
+    "<|citation_start|>",
+    "<|citation_end|>",
+    "<|memory_read_start|>",
+    "<|memory_read_end|>",
+    "<|memory_write_start|>",
+    "<|memory_write_end|>",
+    "<|final_answer_start|>",
+    "<|final_answer_end|>",
+    "<|json_start|>",
+    "<|json_end|>",
+    "<|code_start|>",
+    "<|code_end|>",
+    "<|markdown_start|>",
+    "<|markdown_end|>",
+    "<|duplex_start|>",
+    "<|duplex_end|>",
+    "<|system_channel|>",
+    "<|listen|>",
+    "<|speak|>",
+    "<|listen_speak|>",
+    "<|output_audio_start|>",
+    "<|output_audio_end|>",
+    "<|text_start|>",
+    "<|text_end|>",
+    "<|overlap|>",
+    "<|barge_in|>",
+    "<|interruption|>",
+    "<|interruption_repair|>",
+    "<|backchannel|>",
+    "<|turn_yield|>",
+    "<|hold|>",
+    "<|silence|>",
+    "<|non_speech|>",
+    "<|voice_reference_start|>",
+    "<|voice_reference_end|>",
+    "<|voice_reference|>",
+    "<|voice_switch|>",
+    "<|speaker_style|>",
+    "<|prosody_control|>",
+    "<|zh_tw|>",
+    "<|zh_hant|>",
+    "<|taigi|>",
+    "<|hakka|>",
+    "<|bopomofo|>",
+    "<|mixed_en|>",
+    "<|en|>",
+    "<|ja|>",
+    "<|ko|>",
+    "<|vi|>",
+    "<|id|>",
+    "<|th|>",
+    "<|asr|>",
+    "<|tts|>",
+    "<|speaker_0|>",
+    "<|speaker_1|>",
+    "<|speaker_2|>",
+    "<|speaker_3|>",
+    "<|timestamp|>",
+    "<|noise|>",
+    "<|laugh|>",
+    "<|breath|>",
+    "<|pause|>",
+    "<|prosody|>",
+    "<|pron|>",
+    "</|pron|>",
+    "<|image|>",
+    "<|ocr|>",
+    "<|bbox|>",
+    "<|line|>",
+    "<|table|>",
+    "<|row|>",
+    "<|col|>",
+    "<|cell|>",
+    "<|reading_order|>",
+    "<|source|>",
+    "<|cite|>",
+    "<|evidence|>",
+    "<|quote|>"
+  ],
+  "fix_mistral_regex": true,
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "model_type": "byte_level_bpe",
+  "no_audio_codec_tokens": true,
+  "no_dense_timestamp_tokens": true,
+  "open_formosa": {
+    "required_special_token_count": 157,
+    "required_special_tokens_present": true,
+    "required_special_tokens_single_id": true,
+    "standard_special_tokens": {
+      "bos_token": "<s>",
+      "eos_token": "</s>",
+      "pad_token": "<pad>",
+      "unk_token": "<unk>"
+    }
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "rich_transcription": {
+    "allow_non_speech_events": true,
+    "compact_json": true,
+    "default_format": "json_segments",
+    "enabled": true,
+    "include_content": true,
+    "include_speaker": true,
+    "include_start_end": true,
+    "no_dense_timestamp_tokens": true,
+    "timestamp_precision_digits": 2,
+    "timestamp_unit": "seconds"
+  },
+  "special_tokens": [
+    "<|pad|>",
+    "<|bos|>",
+    "<|eos|>",
+    "<|unk|>",
+    "<|system|>",
+    "<|user_channel|>",
+    "<|assistant_channel|>",
+    "<|task:speech_to_text|>",
+    "<|task:text_to_speech|>",
+    "<|input_audio_start|>",
+    "<|input_audio_end|>",
+    "<|audio_ref_start|>",
+    "<|audio_ref_end|>",
+    "<|audio_start|>",
+    "<|audio_end|>",
+    "<|speech_start|>",
+    "<|speech_end|>",
+    "<|transcript_start|>",
+    "<|transcript_end|>",
+    "<|segment_start|>",
+    "<|segment_end|>",
+    "<|speaker|>",
+    "<|start_time|>",
+    "<|end_time|>",
+    "<|duration|>",
+    "<|content|>",
+    "<|non_speech_event|>",
+    "<|retrieval_result_start|>",
+    "<|retrieval_result_end|>",
+    "<|ocr_start|>",
+    "<|ocr_end|>",
+    "<|image_start|>",
+    "<|image_end|>",
+    "<|video_start|>",
+    "<|video_end|>"
+  ],
+  "strict_no_dense_timestamp_tokens": true,
+  "tokenizer_class": "GPT2Tokenizer",
+  "truncation_side": "right",
+  "unk_token": "<unk>",
+  "vocab_size": 114688
+}