|
Download README.md from codebucketsolutions/edge-punct-casing-en-onnx: direct link, hf CLI and curl.
- Browser
- Download file 3.4 kB
-
https://huggingface.co/codebucketsolutions/edge-punct-casing-en-onnx/resolve/main/README.md
- Command line
-
hf download hf://codebucketsolutions/edge-punct-casing-en-onnx/README.md
-
curl -L -o README.md https://huggingface.co/codebucketsolutions/edge-punct-casing-en-onnx/resolve/main/README.md
3.4 kB
| license: apache-2.0 | |
| library_name: onnx | |
| language: | |
| - en | |
| base_model: frankyoujian/Edge-Punct-Casing | |
| tags: | |
| - punctuation | |
| - truecasing | |
| - sherpa-onnx | |
| - onnx | |
| inference: false | |
| # Edge-Punct-Casing (English), ONNX, for the browser | |
| English punctuation and word casing for transcripts that have neither: a CNN-BiLSTM with a 5,000-piece unigram | |
| vocabulary by Jian You (Cisco Systems), from the paper "A lightweight and efficient punctuation and word casing | |
| prediction model for on-device streaming ASR" ([code](https://github.com/frankyoujian/Edge-Punct-Casing), | |
| [weights](https://huggingface.co/frankyoujian/Edge-Punct-Casing)). It adds commas, full stops and question marks | |
| after words and predicts lower case, capitalised or all-capital words. | |
| This repository holds the two files of the sherpa-onnx package of that model, so that a web page can fetch them | |
| (the package is only published as a `.tar.bz2` archive on GitHub releases, which a page cannot read). It is not | |
| an official release of the author or of sherpa-onnx. | |
| ## What was changed | |
| Nothing in the files. They were taken from | |
| [`sherpa-onnx-online-punct-en-2024-08-06.tar.bz2`](https://github.com/k2-fsa/sherpa-onnx/releases/download/punctuation-models/sherpa-onnx-online-punct-en-2024-08-06.tar.bz2) | |
| (SHA-256 `9f5e5a72c7d2829635bd074fce92b6bbd5b78da8a52e7ad8ed1be933f366b99d`), which sherpa-onnx built from the | |
| author's `sherpa-onnx-cnn-bilstm-unigram-bpe-en.7z`. Only the 8-bit model is copied; the archive's float32 | |
| `model.onnx` (29.7 MB) is left out. | |
| | File | Bytes | SHA-256 | | |
| | --- | --- | --- | | |
| | `model.int8.onnx` | 7,490,500 | `9d611f445fe4a46186080fe161be6059d87d72eb88d3a8cb00c1a06e83a6067e` | | |
| | `bpe.vocab` | 149,430 | `e118b7ad88c54db562517df49e1cffd4836d166c34fb190fd311d7f34eb238f5` | | |
| ## How it runs | |
| The steps are sherpa-onnx's `OnlinePunctuationCNNBiLSTMImpl` and simple-sentencepiece: | |
| 1. Split the text on ASCII white space. Cut each word, with `▁` in front, into vocabulary pieces: the cut with the | |
| highest sum of the pieces' scores (float32), the shorter first piece on a tie, `<unk>` (0) for a byte no piece | |
| starts with. | |
| 2. Chunks of at most 200 tokens: `<s>` (1), the pieces, `</s>` (2), padded with 0. `valid_ids` is 1 for `<s>`, | |
| the first piece of each word and `</s>`, else 0; `label_lens` is the number of ones. | |
| 3. `token_ids` and `valid_ids` int32 `[N, 200]`, `label_lens` int32 `[N]` give `active_case_logits` and | |
| `active_punct_logits`, float32 `[words, 4]`: one row per word, chunks one after the other. | |
| 4. Case (argmax): 0 as written, 1 all capitals, 2 first letter capital, 3 mixed (left as written). Punctuation | |
| (argmax): 0 none, 1 `,`, 2 `.`, 3 `?`, written after the word. Only ASCII letters change case. | |
| A word longer than one chunk crashes sherpa-onnx 2.0.1; keep words under 190 bytes. | |
| [bolo](https://github.com/cB-Sigma/bolo-speech) runs it on its server through sherpa-onnx (`punct-en-cnn-bilstm`) | |
| and in its browser demo with ONNX Runtime Web, with a TypeScript port of these steps that gives the same text as | |
| sherpa-onnx on its test transcripts. | |
| ## Licence and attribution | |
| Apache-2.0 (see `LICENSE`, the licence of [Edge-Punct-Casing](https://github.com/frankyoujian/Edge-Punct-Casing) | |
| and of its weights on the Hugging Face Hub). Model and training by Jian You (Cisco Systems); ONNX packaging by the | |
| [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) project (Apache-2.0). | |