EdgeASR / FFASR.md
frankyoujian's picture
Document offline FFASR setup and pin the custom evaluator
43b3767 verified
|
Raw History Blame Contribute Delete
2.04 kB

FFASR offline evaluation

EdgeASR is an offline/non-streaming English Zipformer RNN-T model. It processes the whole recording with a non-causal encoder. Do not select Streaming ASR for this model.

Submission fields

  • Model ID: frankyoujian/EdgeASR
  • Task: offline ASR (asr in the API).
  • Extra Python requirements: copy the non-comment lines of requirements.txt.
  • Setup script: leave empty.
  • Custom evaluator: copy all of ffasr_custom_evaluator.py.
  • Contact email: provide your own email directly to FFASR; it is not in this repo.
  • Base model: fill only if applicable to the training history.

The custom evaluator pins runtime revision 496a1127c8dd4420e6934f7790cb1ddb33af13b4. The evaluator loads the model once, uses FP32 and greedy RNN-T decoding with one symbol per encoder frame, and exposes the loaded model for parameter counting. Batch size is 1. There is no external LM or dataset-specific prompt/decoding configuration.

Transcript formatting

After BPE reconstruction, <cap> capitalizes the preceding word, <upp> uppercases it, and <.>, <,>, <?> become punctuation. Formatting tags are removed after applying their effects. FFASR applies its own WER normalization after receiving the formatted transcript.

we <cap> are from us <upp> <,> and they are from china <cap> <.> who <cap> are they from <?>
We are from US, and they are from China. Who are they from?

Reproducibility and validation

artifact_manifest.json records SHA-256 hashes of the unchanged checkpoint, token table, BPE model, configuration, formatter, dependencies and source. The source snapshot retains its original license under source/LICENSE.

Sixteen formatter and production text-return tests passed during preparation, including actual model token IDs and the exact example above. Dependencies were resolved for Linux x86-64 / Python 3.12 / CUDA 12.4. Native GPU inference was not executed as part of package preparation. This document makes no claim of an official FFASR result.