---
license: other
license_name: paraformer-upstream
license_link: https://github.com/modelscope/FunASR
language:
- zh
library_name: coreml
tags:
- coreml
- ane
- speech-recognition
- paraformer
- funasr
- fluidaudio
pipeline_tag: automatic-speech-recognition
---

# Paraformer-large (zh) — CoreML (Apple Neural Engine)

CoreML conversion of FunASR's **Paraformer-large** (Mandarin Chinese) for on-device
inference on Apple Silicon, for [FluidInference/FluidAudio](https://github.com/FluidInference/FluidAudio).

Paraformer is a **non-autoregressive** ASR model: a SANM encoder, a CIF predictor
that emits one acoustic-embedding token per output character, and a parallel
(single-pass) decoder. Upstream:
[iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch](https://www.modelscope.cn/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch).

## Files (3 CoreML stages + host CIF)

| File | Precision | Compute unit | Size | Role |
|------|-----------|--------------|------|------|
| `ParaformerPreprocessor.mlmodelc` | FP32 | CPU | 3 MB | front-end: waveform → 560-d LFR features |
| `ParaformerEncoder.mlmodelc` | FP16 | **ANE** | 302 MB | SANM encoder (enumerated buckets `[128,256,512,1024,1800]`) |
| `ParaformerDecoder.mlmodelc` | FP16 | **ANE** | 109 MB | parallel decoder (enc 512, tokens 128) |
| `vocab.json` | — | — | — | 8404 CharTokenizer tokens (array form) |

The **CIF predictor** runs on the host between encoder and decoder (it emits a
*dynamic* token count, which a fixed-shape CoreML graph can't express). It's a
small conv1d + linear + sigmoid → integrate-and-fire; a numpy reference
(`cif_numpy.py`) is in the conversion repo as the Swift blueprint.

## Pipeline

```
waveform → [Preprocessor fp32/CPU] → features [1,T,560]
        → [Encoder fp16/ANE] → enc_out [1,T,512]
        → [host CIF] → acoustic_embeds [1,L,512], token_count L
        → [Decoder fp16/ANE] → logits [1,L,8404]
        → argmax per token → drop sos(1)/eos(2)/blank(0) → CharTokenizer
```

> Both fp16 encoder/decoder are correct on the Neural Engine. The front-end runs
> FP32/CPU (power-spectrum + log exceed the FP16 range). Run the encoder/decoder
> with `MLModelConfiguration.computeUnits = .cpuAndNeuralEngine`.

## Conversion notes

Two SANM-specific fixes were required for fp16/ANE under bucket padding (see the
conversion repo): a fp16-safe attention mask fill (`-inf` → `-1e4`), and building
the encoder/decoder pad-masks from the **input tensor's seq dim** (so
`EnumeratedShapes` generalize) rather than `lengths.max()`.

## Benchmark — AISHELL-1 test (CoreML on ANE)

Full test set (7,176 utts), full-CoreML pipeline on M5 Pro ANE:

| Precision | size (enc+dec) | CER | median RTFx | peak RAM |
|-----------|----------------|-----|-------------|----------|
| fp16 (default) | 411 MB | **2.12%** | 85× | 0.38 GB |
| int8 | 207 MB | **2.12%** | 84× | 0.24 GB |

Official Paraformer-large AISHELL-1 ≈ 1.95% CER (the ~0.17 pp gap is fp16 + the
fixed-shape decoder padding). int8 weight quantization is accuracy-neutral (CER
unchanged), ~half the size/memory.

Reproduces the published Paraformer-large AISHELL-1 number — confirming the
conversion (front-end + encoder + CIF + decoder) is faithful.

## License & attribution

Weights derive from FunASR's Paraformer-large; the upstream license applies. This
repo is a format conversion only (no retraining). See
[FunASR](https://github.com/modelscope/FunASR).