EdgeASR / FFASR.md
frankyoujian's picture
Document offline FFASR setup and pin the custom evaluator
43b3767 verified
|
Raw History Blame Contribute Delete
2.04 kB
# FFASR offline evaluation
EdgeASR is an **offline/non-streaming English Zipformer RNN-T model**.
It processes the whole recording with a non-causal encoder. Do not select
Streaming ASR for this model.
## Submission fields
- Model ID: `frankyoujian/EdgeASR`
- Task: offline ASR (`asr` in the API).
- Extra Python requirements: copy the non-comment lines of `requirements.txt`.
- Setup script: leave empty.
- Custom evaluator: copy all of `ffasr_custom_evaluator.py`.
- Contact email: provide your own email directly to FFASR; it is not in this repo.
- Base model: fill only if applicable to the training history.
The custom evaluator pins runtime revision `496a1127c8dd4420e6934f7790cb1ddb33af13b4`. The evaluator loads
the model once, uses FP32 and greedy RNN-T decoding with one symbol per encoder
frame, and exposes the loaded model for parameter counting. Batch size is 1.
There is no external LM or dataset-specific prompt/decoding configuration.
## Transcript formatting
After BPE reconstruction, `<cap>` capitalizes the preceding word, `<upp>`
uppercases it, and `<.>`, `<,>`, `<?>` become punctuation. Formatting tags are
removed after applying their effects. FFASR applies its own WER normalization
after receiving the formatted transcript.
```text
we <cap> are from us <upp> <,> and they are from china <cap> <.> who <cap> are they from <?>
```
```text
We are from US, and they are from China. Who are they from?
```
## Reproducibility and validation
`artifact_manifest.json` records SHA-256 hashes of the unchanged checkpoint,
token table, BPE model, configuration, formatter, dependencies and source.
The source snapshot retains its original license under `source/LICENSE`.
Sixteen formatter and production text-return tests passed during preparation,
including actual model token IDs and the exact example above. Dependencies
were resolved for Linux x86-64 / Python 3.12 / CUDA 12.4. Native GPU inference
was not executed as part of package preparation. This document makes no
claim of an official FFASR result.