Add the sherpa-onnx Paraformer export repackaged for onnx-asr

e580070 verified 3 days ago

3.18 kB

license: apache-2.0
language:
  - zh
library_name: onnx-asr
pipeline_tag: automatic-speech-recognition
tags:
  - automatic-speech-recognition
  - onnx
  - onnx-asr
  - paraformer
  - funasr
base_model:
  - csukuangfj/sherpa-onnx-paraformer-zh-small-2024-03-09

Paraformer small ZH ONNX

The small Mandarin Paraformer for onnx-asr, about a tenth of the size of the large model. The upstream repository ships only the int8 graph, so load it with quantization="int8".

What this is

Paraformer is the Alibaba FunASR offline non-autoregressive recognizer. A SAN-M encoder reads the audio, a CIF predictor decides how many tokens the utterance has, and a single pass decoder emits all of them at once. There is no decoding loop, so one forward pass gives the transcript.

The ONNX graphs here are copied byte for byte from the sherpa-onnx exports by csukuangfj. Only the side files changed: tokens.txt became vocab.txt, and config.json carries the FunASR frontend statistics from am.mvn.

Usage

The paraformer model type is on the feat/paraformer branch of the TigreGotico onnx-asr fork.

pip install "onnx-asr[cpu,hub] @ git+https://github.com/TigreGotico/onnx-asr@feat/paraformer"

import onnx_asr

model = onnx_asr.load_model("OpenVoiceOS/paraformer-zh-small-onnx", quantization="int8")
print(model.recognize("audio.wav"))

Graph contract

Item	Value
Input	`speech`, float32, `[batch, num_frames, 560]`
Input	`speech_lengths`, int32, `[batch]`
Output	`logits`, float32, `[batch, num_tokens, 8359]`
Output	`token_num`, int32, `[batch]`, the CIF token count

The 560 dim input is the FunASR frontend: an 80 dim kaldi fbank of a waveform scaled to the int16 range, then a low frame rate stack of 7 frames with a hop of 6, then the am.mvn mean variance statistics. onnx-asr computes the fbank with its wespeaker preprocessor and applies the LFR stack and the CMVN in the runtime. Decoding is one argmax per logits row, stopping at </s> and never reading past token_num.

Streaming

A streaming Paraformer also exists upstream. It uses a different graph with encoder and decoder states and needs a streaming runtime, which onnx-asr does not have yet (upstream issue #21). Only the offline model is mirrored here.

Attribution and license

Model: Alibaba DAMO Academy / FunASR, speech_paraformer_asr_nat-zh-cn-16k-common-vocab8358-onnx, Apache-2.0.
ONNX export: sherpa-onnx by Fangjun Kuang (csukuangfj), Xiaomi.
This repository only repackages those files for onnx-asr.

Files

model_int8.onnx (82 MB). There is no fp32 graph upstream.

Parity with native FunASR

4 FLEURS clips, native FunASR on the same source checkpoint with dither = 0.

Clip	int8
zh_1	identical to the large model
zh_2	one character differs from the large model (银合金 / 银和金)