Release Hojo-TTS-Light-40M AX650/NPU3: C++ SDK + axmodels (host-KV LM, whisper-style U16 config)

README.md

@@ -0,0 +1,31 @@
+# Hojo-TTS-Light-40M — AX650/NPU3 预编译模型 + C++ SDK
+
+Hojo-TTS-Light-40M 中文/英文 TTS（24kHz，15 音色）在爱芯 AX650/NPU3 上的预编译部署包。
+全链路 NPU：LM（ax-llm s4/W4A16）+ fine_local + decoder。
+
+## 内容
+- `models/`：预编译模型
+  - `lm_v1_60fix/`：LM 10 层 decode-only axmodel + post + embedding（Pulsar2 6.0 llm_build，s4/W4A16）
+  - `fine_local.axmodel`（INT8，cos 0.9997）
+  - `decoder_sq.axmodel`（SmoothQuant+U16，mag 0.998+ / phase 0.95-0.97）
+  - 音色 / 词典 / speaker_vecs / id2code / speaker_embeds
+- `bin/`：C++ 可执行（AX650/aarch64）
+  - `hojo_tts_text`：纯 C++ 文本 → wav（内部完成 tokenizer + embedding 预处理）
+  - `hojo_tts_cpp` / `tts_driver`：调试用
+
+## 用法（板端）
+```bash
+./bin/hojo_tts_text \
+  models models/lm_v1_60fix/embed_tokens.bin models/speaker_embeds.bin \
+  models/lm_v1_60fix models/fine_local.axmodel models/decoder_sq.axmodel \
+  models/speaker_vecs.bin 0 models/id2code.bin \
+  "Hello, this is a test of the Hojo TTS system." voice_0 out.wav
+```
+输出 `out.wav`（24kHz）。音色索引 0-14（voice_ids 见 models/Hojo-TTS-Light-40M-voice.npz）。
+
+## 精度
+- LM step0 logits cos 0.972（vs fp32 ONNX），top1 精确命中；decode ~162 tok/s
+- 全链路 RTF≈0.4；端到端音频正常
+
+## 复现
+模型转换与 C++ SDK 构建源码见 GitHub：`ml-inory/hojo-tts-light.axera`

bin/hojo_tts_cpp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28e4402aa3aec13384b5b2654b6ed68ddfa9ff990643bbbd8e41244b7ab88fae
+size 1749984

bin/hojo_tts_text

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:787cfe80e303a2ed31c0cd73c31fe16147aca9872471049415cfe0752768d083
+size 3100608

bin/tts_driver

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad8d073f8988ab8c48f59777528cb94f4918e655728b0fcf7946a5530f9e2325
+size 1735680

models/Hojo-TTS-Light-40M-voice.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:048c23d0db973298cfb1e8d1d079b079ca039f656b35bd6298bc20cef0f43a3f
+size 36724150

models/README.md

@@ -0,0 +1,12 @@
+# 模型资产
+
+| 组件 | 文件 | 说明 |
+|------|------|------|
+| LM (10 层 decode-only) | qwen3_p8_l0..9_together.axmodel + qwen3_post.axmodel + embed_tokens.bin + config.json + post_config.json + tokenizer.* | ax-llm s4(W4A16)，Pulsar2 6.0 llm_build v1，kv_cache_len=2176 |
+| fine_local | fine_local.axmodel | INT8，cos 0.9997 |
+| decoder | decoder_sq.axmodel | SmoothQuant+U16，mag 0.998+/phase 0.95-0.97（接受） |
+| 词典 | tokenizer.json / tokenizer_config.json | 原版 Hojo |
+| 音色 | Hojo-TTS-Light-40M-voice.npz | token_embedding + speaker_embeds/vecs（15 音色） |
+
+板端部署路径示例：LM → /mnt/sdcard/hojo_sim/hojo_axllm/model_v1_60fix，
+其余 → /mnt/sdcard/hojo_e2e。

models/decoder_sq.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34c132fc5d6d33a2a8b2338b7a16a440f749bcb4e3f03e8b0ba9a49ccd2a67f7
+size 36727167

models/fine_local.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e77842c02f3b04764576fb0f27778d1e0e706daf88eb942a12a7d4a4306226f8
+size 32874686

models/id2code.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff4b7b9888c8ebb01ee865ac6f31086713e8e3362fa05f49ac24f95249b1b377
+size 141480

models/lm_v1_60fix/config.json

@@ -0,0 +1,20 @@
+{
+  "model_name": "hojo-tts-light-40m-lm",
+  "tokenizer_type": "qwen3",
+  "url_tokenizer_model": "tokenizer.json",
+  "template_filename_axmodel": "qwen3_p8_l%d_together.axmodel",
+  "axmodel_num": 10,
+  "filename_post_axmodel": "qwen3_post.axmodel",
+  "filename_tokens_embed": "embed_tokens.bin",
+  "tokens_embed_num": 17685,
+  "tokens_embed_size": 512,
+  "prefill_token_num": 8,
+  "prefill_max_token_num": 32,
+  "kv_cache_num": 2048,
+  "kv_cache_size": 128,
+  "max_token_len": 2048,
+  "post_config_path": "post_config.json",
+  "b_bos": false,
+  "b_eos": false,
+  "use_mmap_load_embed": true
+}

models/lm_v1_60fix/embed_tokens.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75f69ce61bcb60a9bd9ece10ba7004e5b7fdf8adf8a01de532a1be3969b98947
+size 18109440

models/lm_v1_60fix/post_config.json

@@ -0,0 +1,11 @@
+{
+  "enable_repetition_penalty": false,
+  "repetition_penalty": 1.0,
+  "penalty_window": 20,
+  "enable_temperature": true,
+  "temperature": 0.0,
+  "enable_top_k_sampling": false,
+  "top_k": 1,
+  "enable_top_p_sampling": false,
+  "top_p": 1.0
+}

models/lm_v1_60fix/qwen3_p8_l0_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00da80e558a2f81f2e59d3fe35d67394e61bf129e8a2355459cd4e66d099a01c
+size 3512227

models/lm_v1_60fix/qwen3_p8_l1_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e34f6e42f788a980299eab021189de079f6aae2504965d33dac0243b4cc4491
+size 3512227

models/lm_v1_60fix/qwen3_p8_l2_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ca0bc2d5ff6dc5860f38df5b14ae45d35e879e88fb71a87dbe0bbb7870101e6
+size 3512227

models/lm_v1_60fix/qwen3_p8_l3_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69ebe92f48eafeedc9e8c745f56d6a5668bc07d73d9671585d6bc7299a5c8290
+size 3512227

models/lm_v1_60fix/qwen3_p8_l4_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:868e991984edabc66c27a03c8982f7c7632a9bb9145610cc96b4cb5965b60d73
+size 3512227

models/lm_v1_60fix/qwen3_p8_l5_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc4160f5eaaca12e1cb083c6521f410ff3f66764fbdc09f74529a69e8b0dca94
+size 3512227

models/lm_v1_60fix/qwen3_p8_l6_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8338f7c122fd08fe4d31f513fc203e6efa7f6a87b2ff821566d1066080b309b8
+size 3512227

models/lm_v1_60fix/qwen3_p8_l7_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62b794d77d0bdc8cc50988be7fd61ea2ea868bce185712d939f9f401cf4b7efd
+size 3512227

models/lm_v1_60fix/qwen3_p8_l8_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71cfb46b41ebe0535392e1e7e406f45a345ad19e3b76e340957b88d58cbdf782
+size 3512227

models/lm_v1_60fix/qwen3_p8_l9_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1301ba18807d3038284cef18d69eee2af45b86db8c64388a81bb4fe9f283f430
+size 3512227

models/lm_v1_60fix/qwen3_post.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c362379210a8da051ead09bf87c8711b94895607c0af67f139400b61eac2f498
+size 9883094

models/lm_v1_60fix/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "[START]",
+  "bos_token_id": 2,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "[STOP]",
+  "eos_token_id": 3,
+  "is_local": true,
+  "max_length": null,
+  "model_max_length": 4096,
+  "pad_to_multiple_of": null,
+  "pad_token": "[STOP]",
+  "pad_token_id": 3,
+  "pad_token_type_id": 0,
+  "padding_side": "left",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "[UNK]",
+  "unk_token_id": 0
+}

models/model_meta.json

@@ -0,0 +1,205 @@
+{
+  "model_name": "hojo-tts-light-40m",
+  "task": "text-to-speech (24kHz, 15 voices), 3 模型流水线",
+  "note": "LM 为 host-KV 逐 token decode；fine_local/decoder 静态 T_MAX=2048（decoder 复制末行 padding 后截断）",
+  "models": {
+    "lm_decode": {
+      "file": "export/lm_decode.onnx",
+      "inputs": [
+        {
+          "name": "inputs_embeds",
+          "shape": [
+            1,
+            1,
+            512
+          ],
+          "dtype": "float32",
+          "layout": "NCT",
+          "description": "token_embedding 查表 + 说话人 16 槽注入"
+        },
+        {
+          "name": "position_ids",
+          "shape": [
+            1,
+            1
+          ],
+          "dtype": "int64",
+          "layout": "N",
+          "description": "当前 token 绝对位置"
+        },
+        {
+          "name": "attention_mask",
+          "shape": [
+            1,
+            1,
+            1,
+            2049
+          ],
+          "dtype": "float32",
+          "layout": "NCHW",
+          "description": "0=有效槽(0..p-1 与槽2048), -3.38953e38=padding"
+        },
+        {
+          "name": "past_key_values.{0..9}.key/value",
+          "shape": [
+            1,
+            1,
+            2048,
+            128
+          ],
+          "dtype": "float32",
+          "layout": "NCHW",
+          "description": "host KV cache（compact 前缀，尾随零槽）"
+        }
+      ],
+      "outputs": [
+        {
+          "name": "logits",
+          "shape": [
+            1,
+            1,
+            17685
+          ],
+          "dtype": "float32",
+          "description": "下一 token 分布"
+        },
+        {
+          "name": "last_hidden_state",
+          "shape": [
+            1,
+            1,
+            512
+          ],
+          "dtype": "float32"
+        },
+        {
+          "name": "present.{0..9}.key/value",
+          "shape": [
+            1,
+            1,
+            2049,
+            128
+          ],
+          "dtype": "float32",
+          "description": "新 KV 在槽 2048（host 拷回缓存槽 p+1）"
+        }
+      ]
+    },
+    "fine_local": {
+      "file": "export/fine_local.onnx",
+      "inputs": [
+        {
+          "name": "hidden_states",
+          "shape": [
+            1,
+            2048,
+            512
+          ],
+          "dtype": "float32",
+          "layout": "NTC",
+          "description": "LM last_hidden[audio_pos-1]，尾部零 padding"
+        },
+        {
+          "name": "coarse_embeddings",
+          "shape": [
+            1,
+            2048,
+            512
+          ],
+          "dtype": "float32",
+          "layout": "NTC",
+          "description": "token_embedding[coarse_token]，尾部零 padding"
+        },
+        {
+          "name": "speaker_embedding",
+          "shape": [
+            1,
+            192
+          ],
+          "dtype": "float32",
+          "layout": "NC"
+        },
+        {
+          "name": "valid_mask",
+          "shape": [
+            1,
+            2048
+          ],
+          "dtype": "bool",
+          "layout": "N"
+        }
+      ],
+      "outputs": [
+        {
+          "name": "binary_logits",
+          "shape": [
+            1,
+            2048,
+            128
+          ],
+          "dtype": "float32",
+          "description": ">0 -> +1, else -1（只取前 T 行）"
+        }
+      ]
+    },
+    "decoder": {
+      "file": "export/decoder.onnx",
+      "inputs": [
+        {
+          "name": "bits",
+          "shape": [
+            2048,
+            128
+          ],
+          "dtype": "float32",
+          "layout": "TC",
+          "description": "+-1 bits，尾部复制末行 padding"
+        }
+      ],
+      "outputs": [
+        {
+          "name": "mag",
+          "shape": [
+            961,
+            2048
+          ],
+          "dtype": "float32",
+          "description": "频谱幅度（只取前 T 列）"
+        },
+        {
+          "name": "phase",
+          "shape": [
+            961,
+            2048
+          ],
+          "dtype": "float32",
+          "description": "频谱相位（只取前 T 列）"
+        }
+      ]
+    }
+  },
+  "istft": {
+    "n_fft": 1920,
+    "hop_length": 480,
+    "win_length": 1920,
+    "padding": "same"
+  },
+  "sample_rate": 24000,
+  "lm_max_len": 2048,
+  "t_max": 2048,
+  "deployment": {
+    "target": "AX650 / NPU3",
+    "lm": "ax-llm s4(W4A16) decode-only 链: model_v1_60fix/qwen3_p8_l0..9_together.axmodel + qwen3_post.axmodel + embed_tokens.bin (Pulsar2 6.0 llm_build, kv_cache_len=2176)",
+    "fine_local": "compile/fine_local.axmodel (INT8, cos 0.9997)",
+    "decoder": "compile/decoder/decoder_sq.axmodel (SmoothQuant+U16, mag 0.998+/phase 0.95-0.97, 用户接受)",
+    "runtime_fix": "ax-llm LLM.cpp canonical_cap history_cap 修复 (patches/axllm_kv_row_width_fix.patch)",
+    "sdk": "sdk/python/hojo_tts_axllm.py (全链路) + sdk/cpp (tts_driver LM)",
+    "metrics": {
+      "lm_step0_cos": 0.972,
+      "lm_decode_tok_s": 162,
+      "fl_cos": 0.9997,
+      "dec_mag_cos": "0.998-0.9998",
+      "dec_phase_cos": "0.95-0.97 (用户接受)"
+    }
+  }
+}

models/speaker_embeds.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:620c7a0cc34eb10596df654dd8df2c23b712af96b2e373e5130c88634ed9542e
+size 491520

models/speaker_vecs.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc8d312eb7c9eaee4928ecb87ef0d978d6c0aa578b4a425e2d13cd503484f2de
+size 11520

models/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "[START]",
+  "bos_token_id": 2,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "[STOP]",
+  "eos_token_id": 3,
+  "is_local": true,
+  "max_length": null,
+  "model_max_length": 4096,
+  "pad_to_multiple_of": null,
+  "pad_token": "[STOP]",
+  "pad_token_id": 3,
+  "pad_token_type_id": 0,
+  "padding_side": "left",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "[UNK]",
+  "unk_token_id": 0
+}