docs: align spectrogram axes, document checkpoint
Browse files
README.md
CHANGED
|
@@ -1,28 +1,10 @@
|
|
| 1 |
-
---
|
| 2 |
-
library_name: axera
|
| 3 |
-
pipeline_tag: audio-to-audio
|
| 4 |
-
base_model: kiriyamaX/FLowHigh_indep_adaptive_400k
|
| 5 |
-
language:
|
| 6 |
-
- en
|
| 7 |
-
- zh
|
| 8 |
-
tags:
|
| 9 |
-
- axera
|
| 10 |
-
- ax650
|
| 11 |
-
- npu3
|
| 12 |
-
- flowhigh
|
| 13 |
-
- speech-super-resolution
|
| 14 |
-
- audio-super-resolution
|
| 15 |
-
- bandwidth-extension
|
| 16 |
-
- real-time
|
| 17 |
-
- python
|
| 18 |
-
---
|
| 19 |
-
|
| 20 |
# FlowHigh.AXERA
|
| 21 |
|
| 22 |
**8/16 kHz 语音 → 48 kHz 超分 · AX650 NPU3 · Python RTF 0.24（20–60 s 音频）· 2 个 axmodel 共 58 MB · 不做降噪**
|
| 23 |
|
| 24 |
-
  [  [](LICENSE) [](https://huggingface.co/HY-2012/FlowHigh.AXERA)
|
| 6 |
|
| 7 |
+
- HuggingFace: https://huggingface.co/HY-2012/FlowHigh.AXERA
|
| 8 |
- GitHub: https://github.com/ZY-2012/FlowHigh.AXERA
|
| 9 |
- 上游: https://github.com/jjunak-yun/FLowHigh_code
|
| 10 |
|
|
|
|
| 13 |
- 快速开始
|
| 14 |
- 结果
|
| 15 |
- 推理速度
|
| 16 |
+
- 模型转换
|
| 17 |
- 数据来源
|
| 18 |
- License
|
| 19 |
|
|
|
|
| 27 |
|
| 28 |
输入支持 8 kHz / 16 kHz，输出固定 48 kHz。默认 1-step flow、`std=1e-4`；不做降噪。
|
| 29 |
|
| 30 |
+
模型为 `FLowHigh_indep_adaptive_400k`（481,490,826 bytes，Flow 35.4M + BigVGAN 14.0M 参数）。AX650 侧拆成 Flow 38 MB + BigVGAN 20 MB 两个 U16/S8 axmodel；38 MB 是 S8 权重后的正常大小。
|
| 31 |
+
|
| 32 |
## 结果
|
| 33 |
|
| 34 |
### 8 kHz 输入（Open Speech / Samplelib）
|
|
|
|
| 39 |
|
| 40 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 41 |
|---|---|---|
|
| 42 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_us_000_0010_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0010_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_us_000_0010_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0010_output_48k_spec.png" width="360"> |
|
| 43 |
|
| 44 |
**OSR_us_000_0013**
|
| 45 |
|
| 46 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 47 |
|---|---|---|
|
| 48 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_us_000_0013_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0013_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_us_000_0013_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0013_output_48k_spec.png" width="360"> |
|
| 49 |
|
| 50 |
**OSR_us_000_0057**
|
| 51 |
|
| 52 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 53 |
|---|---|---|
|
| 54 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_us_000_0057_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0057_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_us_000_0057_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0057_output_48k_spec.png" width="360"> |
|
| 55 |
|
| 56 |
**OSR_us_000_0059**
|
| 57 |
|
| 58 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 59 |
|---|---|---|
|
| 60 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_us_000_0059_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0059_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_us_000_0059_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_us_000_0059_output_48k_spec.png" width="360"> |
|
| 61 |
|
| 62 |
#### Chinese
|
| 63 |
|
|
|
|
| 65 |
|
| 66 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 67 |
|---|---|---|
|
| 68 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_cn_000_0072_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0072_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_cn_000_0072_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0072_output_48k_spec.png" width="360"> |
|
| 69 |
|
| 70 |
**OSR_cn_000_0073**
|
| 71 |
|
| 72 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 73 |
|---|---|---|
|
| 74 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_cn_000_0073_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0073_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_cn_000_0073_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0073_output_48k_spec.png" width="360"> |
|
| 75 |
|
| 76 |
**OSR_cn_000_0074**
|
| 77 |
|
| 78 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 79 |
|---|---|---|
|
| 80 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_cn_000_0074_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0074_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_cn_000_0074_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0074_output_48k_spec.png" width="360"> |
|
| 81 |
|
| 82 |
**OSR_cn_000_0075**
|
| 83 |
|
| 84 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 85 |
|---|---|---|
|
| 86 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/OSR_cn_000_0075_input_8k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0075_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/OSR_cn_000_0075_output_48k.wav)<br><img src="samples/listen/spectrograms/OSR_cn_000_0075_output_48k_spec.png" width="360"> |
|
| 87 |
|
| 88 |
#### Samplelib（1 分钟，频谱图为前 10 s）
|
| 89 |
|
|
|
|
| 91 |
|
| 92 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 93 |
|---|---|---|
|
| 94 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/samplelib_speech_1m_input_8k.wav)<br><img src="samples/listen/spectrograms/samplelib_speech_1m_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/samplelib_speech_1m_output_48k.wav)<br><img src="samples/listen/spectrograms/samplelib_speech_1m_output_48k_spec.png" width="360"> |
|
| 95 |
|
| 96 |
### 16 kHz 参照（人声）
|
| 97 |
|
|
|
|
| 101 |
|
| 102 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 103 |
|---|---|---|
|
| 104 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/speech_2_input_8k.wav)<br><img src="samples/listen/spectrograms/speech_2_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/speech_2_output_48k.wav)<br><img src="samples/listen/spectrograms/speech_2_output_8k_48k_spec.png" width="360"> |
|
| 105 |
+
| 16k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/16k/speech_2_input_16k.wav)<br><img src="samples/listen/spectrograms/speech_2_input_16k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from16k/speech_2_output_48k.wav)<br><img src="samples/listen/spectrograms/speech_2_output_16k_48k_spec.png" width="360"> |
|
| 106 |
|
| 107 |
**speech_3**
|
| 108 |
|
| 109 |
| 版本 | 输入 | FlowHigh 输出 |
|
| 110 |
|---|---|---|
|
| 111 |
+
| 8k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/8k/speech_3_input_8k.wav)<br><img src="samples/listen/spectrograms/speech_3_input_8k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from8k/speech_3_output_48k.wav)<br><img src="samples/listen/spectrograms/speech_3_output_8k_48k_spec.png" width="360"> |
|
| 112 |
+
| 16k | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/16k/speech_3_input_16k.wav)<br><img src="samples/listen/spectrograms/speech_3_input_16k_spec.png" width="360"> | [▶](https://cdn.jsdelivr.net/gh/ZY-2012/FlowHigh.AXERA@master/samples/listen/48k_from16k/speech_3_output_48k.wav)<br><img src="samples/listen/spectrograms/speech_3_output_16k_48k_spec.png" width="360"> |
|
| 113 |
|
| 114 |
## 推理速度：RTF
|
| 115 |
|
|
|
|
| 119 |
|
| 120 |
20–60 s 音频，1-step flow，BigVGAN 1 s 分块、0.2 s overlap。5 s 短音频因分块/启动开销会略高；C++ 运行时尚未提供。
|
| 121 |
|
| 122 |
+
## 模型转换
|
| 123 |
+
|
| 124 |
+
```bash
|
| 125 |
+
python model_convert/export_flowhigh.py --frames 500 --skip-vocoder --out-dir models
|
| 126 |
+
python model_convert/export_flowhigh.py --frames 100 --skip-flow --out-dir models
|
| 127 |
+
python model_convert/make_calib.py --out-dir build/calib --clips 50
|
| 128 |
+
python model_convert/prepare_compile.py
|
| 129 |
+
PULSAR2_IMAGE=docker-registry.aitsw.axera-tech.com/pulsar2:8.0 \
|
| 130 |
+
bash model_convert/compile.sh flowhigh_nct bigvgan_T100
|
| 131 |
+
```
|
| 132 |
+
|
| 133 |
+
Flow NPU 输入为 `[1,256,500]`（channel-first），图内转置到 `[1,500,256]`；BigVGAN 按 1 s 编译，运行时 overlap 交叉淡化。详见 `model_convert/README.md`。
|
| 134 |
+
|
| 135 |
## 数据来源
|
| 136 |
|
| 137 |
- Open Speech Repository（English / Chinese 8k）：http://voiptroubleshooter.com/open_speech/index.html
|