File size: 3,401 Bytes
a674d07
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
---
license: apache-2.0
library_name: onnx
language:
- en
base_model: frankyoujian/Edge-Punct-Casing
tags:
- punctuation
- truecasing
- sherpa-onnx
- onnx
inference: false
---

# Edge-Punct-Casing (English), ONNX, for the browser

English punctuation and word casing for transcripts that have neither: a CNN-BiLSTM with a 5,000-piece unigram
vocabulary by Jian You (Cisco Systems), from the paper "A lightweight and efficient punctuation and word casing
prediction model for on-device streaming ASR" ([code](https://github.com/frankyoujian/Edge-Punct-Casing),
[weights](https://huggingface.co/frankyoujian/Edge-Punct-Casing)). It adds commas, full stops and question marks
after words and predicts lower case, capitalised or all-capital words.

This repository holds the two files of the sherpa-onnx package of that model, so that a web page can fetch them
(the package is only published as a `.tar.bz2` archive on GitHub releases, which a page cannot read). It is not
an official release of the author or of sherpa-onnx.

## What was changed

Nothing in the files. They were taken from
[`sherpa-onnx-online-punct-en-2024-08-06.tar.bz2`](https://github.com/k2-fsa/sherpa-onnx/releases/download/punctuation-models/sherpa-onnx-online-punct-en-2024-08-06.tar.bz2)
(SHA-256 `9f5e5a72c7d2829635bd074fce92b6bbd5b78da8a52e7ad8ed1be933f366b99d`), which sherpa-onnx built from the
author's `sherpa-onnx-cnn-bilstm-unigram-bpe-en.7z`. Only the 8-bit model is copied; the archive's float32
`model.onnx` (29.7 MB) is left out.

| File | Bytes | SHA-256 |
| --- | --- | --- |
| `model.int8.onnx` | 7,490,500 | `9d611f445fe4a46186080fe161be6059d87d72eb88d3a8cb00c1a06e83a6067e` |
| `bpe.vocab` | 149,430 | `e118b7ad88c54db562517df49e1cffd4836d166c34fb190fd311d7f34eb238f5` |

## How it runs

The steps are sherpa-onnx's `OnlinePunctuationCNNBiLSTMImpl` and simple-sentencepiece:

1. Split the text on ASCII white space. Cut each word, with `▁` in front, into vocabulary pieces: the cut with the
   highest sum of the pieces' scores (float32), the shorter first piece on a tie, `<unk>` (0) for a byte no piece
   starts with.
2. Chunks of at most 200 tokens: `<s>` (1), the pieces, `</s>` (2), padded with 0. `valid_ids` is 1 for `<s>`,
   the first piece of each word and `</s>`, else 0; `label_lens` is the number of ones.
3. `token_ids` and `valid_ids` int32 `[N, 200]`, `label_lens` int32 `[N]` give `active_case_logits` and
   `active_punct_logits`, float32 `[words, 4]`: one row per word, chunks one after the other.
4. Case (argmax): 0 as written, 1 all capitals, 2 first letter capital, 3 mixed (left as written). Punctuation
   (argmax): 0 none, 1 `,`, 2 `.`, 3 `?`, written after the word. Only ASCII letters change case.

A word longer than one chunk crashes sherpa-onnx 2.0.1; keep words under 190 bytes.

[bolo](https://github.com/cB-Sigma/bolo-speech) runs it on its server through sherpa-onnx (`punct-en-cnn-bilstm`)
and in its browser demo with ONNX Runtime Web, with a TypeScript port of these steps that gives the same text as
sherpa-onnx on its test transcripts.

## Licence and attribution

Apache-2.0 (see `LICENSE`, the licence of [Edge-Punct-Casing](https://github.com/frankyoujian/Edge-Punct-Casing)
and of its weights on the Hugging Face Hub). Model and training by Jian You (Cisco Systems); ONNX packaging by the
[sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) project (Apache-2.0).