| # FlowHigh AX650 quantization report |
|
|
| ## Checkpoint |
|
|
| - Model: `FLowHigh_indep_adaptive_400k` |
| - Checkpoint size: 481,490,826 bytes (460 MiB), includes optimizer state. |
| - Weights: Flow 35.4M + BigVGAN 14.0M = 49.4M parameters (FP32 weights about 198 MB). |
| - Source: official Google Drive folder from the FLowHigh README; local copy from the HF mirror `kiriyamaX/FLowHigh_indep_adaptive_400k`. |
| - The 38 MB Flow axmodel is the S8-quantized 35.4M-parameter flow transformer; the 20 MB BigVGAN axmodel is separate. |
|
|
| Official demo `p364_055` 8 kHz: our PyTorch output vs the official Euler output has mel LSD 0.411 and waveform corr 0.998; the board output vs official has corr 0.997, mel LSD 0.515. This confirms the checkpoint/pipeline, not a wrong download. |
|
|
| ## Target |
|
|
| - Board: AX650N, NPU3, axengine 2.12.0s, compiler 8.0. |
| - Flow: U16 activations / S8 weights, 38 MB. |
| - BigVGAN: U16 activations / S8 weights, 20 MB. |
| - Input: 8 kHz / 16 kHz -> 48 kHz. |
| - No denoising. |
|
|
| ## Graph split |
|
|
| ```text |
| host: resample_poly -> log-mel [1,256,T] -> 1-step flow -> BigVGAN chunks -> STFT merge |
| NPU: flow velocity (37 MB) + BigVGAN 1 s chunks (20 MB) |
| ``` |
|
|
| The flow NPU input is channel-first `[1,256,500]`; the wrapper transposes to `[1,500,256]` internally. This is required: Pulsar2 lowers 3D tensors channel-first, and feeding `[1,500,256]` directly makes attention run on the wrong axis. |
|
|
| ## Numerical checks |
|
|
| | Check | Result | |
| |---|---| |
| | Flow ONNX vs PyTorch velocity | max err 1.5e-4 | |
| | Flow U16/S8 on board vs ONNX | SNR 32.96 dB, corr 0.99977 | |
| | Flow latency, 5 s | 21 ms (after warmup) | |
| | BigVGAN ONNX vs PyTorch | max err 8.2e-6 | |
| | BigVGAN U16/S8 on board vs ONNX, 1 s | SNR 20.55 dB, corr 0.99560 | |
| | BigVGAN latency, 1 s | ~93 ms | |
| | BigVGAN chunk overlap | 0.2 s; boundary jump reduced from 0.44 to 0.005 | |
|
|
| BigVGAN is the sensitive part; its direct waveform SNR is low, but the final speech quality after spectral merging is close to the FP32 pipeline. |
|
|
| ## Full pipeline quality |
|
|
| VCTK `p232_013`, 48 kHz reference. Board uses 1-step flow with `std=1e-4`, BigVGAN 1 s / 0.2 s overlap. Baseline and board use different RNG seeds, so the PESQ/LSD comparison is indicative; flow and BigVGAN SNR rows are strict same-input comparisons. |
|
|
| | Input | Pipeline | PESQ-WB | STOI | Mel LSD | |
| |---|---|---|---:|---:| |
| | 8 kHz | PyTorch FP32 | 3.872 | 0.9981 | 2.323 | |
| | 8 kHz | AX650 U16/S8 | **3.876** | **0.9982** | 2.333 | |
| | 16 kHz | PyTorch FP32 | 4.570 | 1.0000 | 2.315 | |
| | 16 kHz | AX650 U16/S8 | **4.573** | **1.0000** | 2.302 | |
|
|
| ## Speed |
|
|
| 5 s input, Python/axengine, one flow step: |
|
|
| | Stage | Time | |
| |---|---:| |
| | Flow | 33 ms | |
| | BigVGAN (6 x 1 s chunks) | 578 ms | |
| | Host STFT/mel/postprocess | ~600 ms | |
| | Total | 0.94–1.22 s | |
| | RTF | 0.19–0.24 | |
|
|
| The current runtime is Python-only. Moving STFT/postprocess to C++ and using a native flow/vocoder loop is the next optimization; the NPU portion itself is 0.61 s for 5 s. |
|
|
| ## Long audio |
|
|
| The flow graph is fixed at 500 mel frames (5 s). Longer audio is processed in 5 s chunks with 0.5 s overlap and waveform crossfade; BigVGAN uses 1 s chunks with 0.2 s overlap. The 60 s Samplelib input was processed end-to-end with no boundary clicks above the normal waveform derivative. |
|
|
| ## Limitations |
|
|
| - The 5 s flow chunk is a hard graph size; longer context would need a different compiled graph. |
| - No denoising; FLowHigh is BWE only. |
| - C++ runtime is not included in this first quantization milestone. |
|
|
