Automatic Speech Recognition
Transformers
English
niagara-19m-batch
asr
speech
state-space-model
ssm
edge-ai
audio
on-device
real-time
low-power
low-latency
cpu
embedded
custom_code
Eval Results
Instructions to use abr-ai/niagara-19m-batch.en with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use abr-ai/niagara-19m-batch.en with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="abr-ai/niagara-19m-batch.en", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("abr-ai/niagara-19m-batch.en", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Daniel Rasmussen commited on
Commit ·
8f68d03
1
Parent(s): 8da6334
Remove 32b from model name
Browse files- README.md +33 -25
- asr-19m-v2-en-32b-cpu.pt → asr-19m-v2-en-cpu.pt +0 -0
- asr-19m-v2-en-32b-cuda.pt → asr-19m-v2-en-cuda.pt +0 -0
- config.json +2 -2
- config.py +1 -1
- notebook.ipynb +1 -1
README.md
CHANGED
|
@@ -9,7 +9,7 @@ metrics:
|
|
| 9 |
pipeline_tag: automatic-speech-recognition
|
| 10 |
library_name: transformers
|
| 11 |
model-index:
|
| 12 |
-
- name: abr-ai/asr-19m-v2-en
|
| 13 |
results:
|
| 14 |
- task:
|
| 15 |
type: automatic-speech-recognition
|
|
@@ -133,9 +133,9 @@ model-index:
|
|
| 133 |
url: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard
|
| 134 |
---
|
| 135 |
|
| 136 |
-
# ABR's asr-19m-v2-en
|
| 137 |
|
| 138 |
-
The asr-19m-v2-en
|
| 139 |
|
| 140 |
## Usage
|
| 141 |
|
|
@@ -148,11 +148,10 @@ pip install datasets torch torchcodec transformers sentencepiece
|
|
| 148 |
### Automatically instantiate the model
|
| 149 |
|
| 150 |
```
|
| 151 |
-
import torch
|
| 152 |
from datasets import load_dataset
|
| 153 |
from transformers import AutoFeatureExtractor, AutoModel, AutoTokenizer
|
| 154 |
|
| 155 |
-
model_id = "abr-ai/asr-19m-v2-en
|
| 156 |
feature_extractor = AutoFeatureExtractor.from_pretrained(
|
| 157 |
model_id, trust_remote_code=True
|
| 158 |
)
|
|
@@ -169,14 +168,14 @@ dataset = load_dataset("librispeech_asr", "clean", split="test", streaming=True)
|
|
| 169 |
samples = list(dataset.take(3)) # Take 3 examples
|
| 170 |
```
|
| 171 |
|
| 172 |
-
Then
|
| 173 |
|
| 174 |
```
|
| 175 |
audio = samples[0]["audio"]["array"]
|
| 176 |
features = feature_extractor(audio)
|
| 177 |
logits = model(features)
|
| 178 |
transcription = tokenizer.decode_from_logits(logits)
|
| 179 |
-
print(transcription)
|
| 180 |
```
|
| 181 |
|
| 182 |
### Transcribing many audio files
|
|
@@ -202,8 +201,8 @@ This model provides transcribed speech as a string for a given audio sample.
|
|
| 202 |
|
| 203 |
## Model Details
|
| 204 |
|
| 205 |
-
The SSM ASR model is trained for English speech recognition and transcribes audio into text. ABR developed the model to demonstrate small, efficient, real-time, accurate speech recognition can be performed with SSMs and run on low cost third party hardware.
|
| 206 |
-
The model uses 19m parameters. The version posted here is a non-causal model (like most on the leaderboard), to give fair performance comparisons. It is also available as a cascaded model, meaning it produces extremely low latency (<120ms from first audio to token) causal outputs as well as 1s latency non-causal outputs. These two streams can be merged to have a quick response that updates
|
| 207 |
|
| 208 |
Release Date: November 18, 2025
|
| 209 |
|
|
@@ -213,7 +212,7 @@ Automatic speech recognition model transcribing speech audio to text in English.
|
|
| 213 |
|
| 214 |
### Model Use
|
| 215 |
|
| 216 |
-
The intended use of the model is for evaluation by AI developers who want extremely small but performant ASR. We recognize that it is not possible to enforce our intended use guidelines. The models should not be used to transcribe individuals without their explicit consent, or be used to infer any particular human features as only text output is generated by the model. Other capabilities have not been evaluated. We recommend against using the model in high-risk settings (such as making important decisions) where errors in the model output can result in significant consequences for users. We strongly recommend that users perform extensive evaluations for their use cases.
|
| 217 |
|
| 218 |
## Training
|
| 219 |
|
|
@@ -221,15 +220,16 @@ The model was trained on datasets partially listed below. It uses MFCC preproces
|
|
| 221 |
|
| 222 |
### Datasets
|
| 223 |
|
| 224 |
-
The datasets
|
| 225 |
-
|
| 226 |
-
-
|
| 227 |
-
-
|
| 228 |
-
-
|
| 229 |
-
-
|
| 230 |
-
-
|
| 231 |
-
-
|
| 232 |
-
-
|
|
|
|
| 233 |
- SPGISpeech
|
| 234 |
|
| 235 |
## Performance
|
|
@@ -252,12 +252,20 @@ Average WER = 10.61 %
|
|
| 252 |
| TED-LIUM | 7.53% |
|
| 253 |
| VoxPopuli | 9.88% |
|
| 254 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 255 |
## Limitations
|
| 256 |
|
| 257 |
-
Since this model was trained on publicly available speech datasets, the performance of this model might degrade for speech which includes technical terms, significant noise, or vernacular that the model has not been trained on. The model might also perform worse for accented speech. The model might generate text that was not actually spoken in the input audio.
|
| 258 |
-
|
| 259 |
-
|
| 260 |
-
|
|
|
|
| 261 |
|
| 262 |
## License
|
| 263 |
|
|
@@ -268,10 +276,10 @@ This model is made available under [ABR's open license](https://www.appliedbrain
|
|
| 268 |
@misc{
|
| 269 |
AppliedBrainResearch2025,
|
| 270 |
author = {Applied Brain Research, Inc},
|
| 271 |
-
title = {asr-19m-v2-en
|
| 272 |
year = {2025},
|
| 273 |
publisher = {HuggingFace},
|
| 274 |
journal = {HuggingFace repository},
|
| 275 |
-
howpublished = {\url{https://huggingface.co/abr-ai/asr-19m-v2-en
|
| 276 |
}
|
| 277 |
```
|
|
|
|
| 9 |
pipeline_tag: automatic-speech-recognition
|
| 10 |
library_name: transformers
|
| 11 |
model-index:
|
| 12 |
+
- name: abr-ai/asr-19m-v2-en
|
| 13 |
results:
|
| 14 |
- task:
|
| 15 |
type: automatic-speech-recognition
|
|
|
|
| 133 |
url: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard
|
| 134 |
---
|
| 135 |
|
| 136 |
+
# ABR's asr-19m-v2-en SSM
|
| 137 |
|
| 138 |
+
The asr-19m-v2-en model is a [State Space Model](https://huggingface.co/blog/lbourdois/get-on-the-ssm-train) (SSM) with attention that performs automatic speech recognition (ASR), trained and released by [Applied Brain Research](http://www.appliedbrainresearch.com) (ABR). This model contains \~19m parameters, transcribes speech in English, and was trained on 15k hours of speech data (competitors use about 200k hours). SSMs are an ideal solution for streaming contexts, but to provide a more direct comparison with other models in the leaderboard this model is not streaming. Variants of this model for real-time streaming and other languages are available under commercial license from [ABR](https://www.appliedbrainresearch.com/state-space-models).
|
| 139 |
|
| 140 |
## Usage
|
| 141 |
|
|
|
|
| 148 |
### Automatically instantiate the model
|
| 149 |
|
| 150 |
```
|
|
|
|
| 151 |
from datasets import load_dataset
|
| 152 |
from transformers import AutoFeatureExtractor, AutoModel, AutoTokenizer
|
| 153 |
|
| 154 |
+
model_id = "abr-ai/asr-19m-v2-en"
|
| 155 |
feature_extractor = AutoFeatureExtractor.from_pretrained(
|
| 156 |
model_id, trust_remote_code=True
|
| 157 |
)
|
|
|
|
| 168 |
samples = list(dataset.take(3)) # Take 3 examples
|
| 169 |
```
|
| 170 |
|
| 171 |
+
Then run the model:
|
| 172 |
|
| 173 |
```
|
| 174 |
audio = samples[0]["audio"]["array"]
|
| 175 |
features = feature_extractor(audio)
|
| 176 |
logits = model(features)
|
| 177 |
transcription = tokenizer.decode_from_logits(logits)
|
| 178 |
+
print(transcription[0])
|
| 179 |
```
|
| 180 |
|
| 181 |
### Transcribing many audio files
|
|
|
|
| 201 |
|
| 202 |
## Model Details
|
| 203 |
|
| 204 |
+
The SSM ASR model is trained for English speech recognition and transcribes audio into text. ABR developed the model to demonstrate small, efficient, real-time, accurate speech recognition can be performed with SSMs and run on low cost third party hardware.
|
| 205 |
+
The model uses 19m parameters. The version posted here is a non-causal model (like most on the leaderboard), to give fair performance comparisons. It is also available as a cascaded model, meaning it produces extremely low latency (\<120ms from first audio to token) causal outputs as well as 1s latency non-causal outputs. These two streams can be merged to have a quick response that updates after 1s with the final result.
|
| 206 |
|
| 207 |
Release Date: November 18, 2025
|
| 208 |
|
|
|
|
| 212 |
|
| 213 |
### Model Use
|
| 214 |
|
| 215 |
+
The intended use of the model is for evaluation by AI developers who want extremely small but performant ASR. We recognize that it is not possible to enforce our intended use guidelines. The models should not be used to transcribe individuals without their explicit consent, or be used to infer any particular human features as only text output is generated by the model. Other capabilities have not been evaluated. We recommend against using the model in high-risk settings (such as making important decisions) where errors in the model output can result in significant consequences for users. We strongly recommend that users perform extensive evaluations for their use cases.
|
| 216 |
|
| 217 |
## Training
|
| 218 |
|
|
|
|
| 220 |
|
| 221 |
### Datasets
|
| 222 |
|
| 223 |
+
The training datasets contain \~15k hours of English speech, including:
|
| 224 |
+
|
| 225 |
+
- LibriSpeech (clean)
|
| 226 |
+
- VoxPopuli
|
| 227 |
+
- GigaSpeech
|
| 228 |
+
- Common Voice
|
| 229 |
+
- TED-LIUM
|
| 230 |
+
- Europarl
|
| 231 |
+
- Earnings-22
|
| 232 |
+
- AMI-IHM
|
| 233 |
- SPGISpeech
|
| 234 |
|
| 235 |
## Performance
|
|
|
|
| 252 |
| TED-LIUM | 7.53% |
|
| 253 |
| VoxPopuli | 9.88% |
|
| 254 |
|
| 255 |
+
## Related models
|
| 256 |
+
|
| 257 |
+
| Model | Parameters | Average WER |
|
| 258 |
+
|:--------------------------------------------------------------------| :---- | :---- |
|
| 259 |
+
| [abr-ai/asr-19m-v2-en](https://huggingface.co/abr-ai/asr-19m-v2-en) | 19M | 10.61% |
|
| 260 |
+
| [abr-ai/asr-38m-v1-en](https://huggingface.co/abr-ai/asr-38m-v1-en) | 38M | 8.84% |
|
| 261 |
+
|
| 262 |
## Limitations
|
| 263 |
|
| 264 |
+
Since this model was trained on publicly available speech datasets, the performance of this model might degrade for speech which includes technical terms, significant noise, or vernacular that the model has not been trained on. The model might also perform worse for accented speech. The model might generate text that was not actually spoken in the input audio.
|
| 265 |
+
|
| 266 |
+
## Broader Implications
|
| 267 |
+
|
| 268 |
+
We intend for the SSM ASR model to be used for beneficial purposes, including low cost transcription on low cost hardware, providing accessibility, and improving real-time voice user interfaces. As with all AI technology, there are also reasons to be concerned about dual-use. For instance, lowering the cost may allow broader deployment of undesired surveillance technology or the inexpensive scaling of existing technology. Related safety concerns come from the model being used to identify individuals or being deployable in very small footprint hardware.
|
| 269 |
|
| 270 |
## License
|
| 271 |
|
|
|
|
| 276 |
@misc{
|
| 277 |
AppliedBrainResearch2025,
|
| 278 |
author = {Applied Brain Research, Inc},
|
| 279 |
+
title = {asr-19m-v2-en},
|
| 280 |
year = {2025},
|
| 281 |
publisher = {HuggingFace},
|
| 282 |
journal = {HuggingFace repository},
|
| 283 |
+
howpublished = {\url{https://huggingface.co/abr-ai/asr-19m-v2-en}},
|
| 284 |
}
|
| 285 |
```
|
asr-19m-v2-en-32b-cpu.pt → asr-19m-v2-en-cpu.pt
RENAMED
|
File without changes
|
asr-19m-v2-en-32b-cuda.pt → asr-19m-v2-en-cuda.pt
RENAMED
|
File without changes
|
config.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
"architectures": [
|
| 3 |
-
"asr-19m-v2-en
|
| 4 |
],
|
| 5 |
"model_type": "asr-19m-v2",
|
| 6 |
"auto_map": {
|
|
@@ -11,7 +11,7 @@
|
|
| 11 |
},
|
| 12 |
"input_features": 80,
|
| 13 |
"vocab_size": 256,
|
| 14 |
-
"torchscript_model_file": "asr-19m-v2-en
|
| 15 |
"tokenizer_file": "sentencepiece_256.model",
|
| 16 |
"sample_rate": 16000,
|
| 17 |
"transformers_version": "4.36.0"
|
|
|
|
| 1 |
{
|
| 2 |
"architectures": [
|
| 3 |
+
"asr-19m-v2-en"
|
| 4 |
],
|
| 5 |
"model_type": "asr-19m-v2",
|
| 6 |
"auto_map": {
|
|
|
|
| 11 |
},
|
| 12 |
"input_features": 80,
|
| 13 |
"vocab_size": 256,
|
| 14 |
+
"torchscript_model_file": "asr-19m-v2-en",
|
| 15 |
"tokenizer_file": "sentencepiece_256.model",
|
| 16 |
"sample_rate": 16000,
|
| 17 |
"transformers_version": "4.36.0"
|
config.py
CHANGED
|
@@ -26,7 +26,7 @@ class Config(PretrainedConfig):
|
|
| 26 |
self,
|
| 27 |
input_features=80,
|
| 28 |
vocab_size=256,
|
| 29 |
-
torchscript_model_file="asr-19m-v2-en
|
| 30 |
tokenizer_file="sentencepiece_256.model",
|
| 31 |
sample_rate=16000,
|
| 32 |
**kwargs
|
|
|
|
| 26 |
self,
|
| 27 |
input_features=80,
|
| 28 |
vocab_size=256,
|
| 29 |
+
torchscript_model_file="asr-19m-v2-en",
|
| 30 |
tokenizer_file="sentencepiece_256.model",
|
| 31 |
sample_rate=16000,
|
| 32 |
**kwargs
|
notebook.ipynb
CHANGED
|
@@ -56,7 +56,7 @@
|
|
| 56 |
"import torch\n",
|
| 57 |
"from transformers import AutoFeatureExtractor, AutoModel, AutoTokenizer\n",
|
| 58 |
"\n",
|
| 59 |
-
"model_id = \"abr-ai/asr-19m-v2-en
|
| 60 |
"feature_extractor = AutoFeatureExtractor.from_pretrained(\n",
|
| 61 |
" model_id, trust_remote_code=True\n",
|
| 62 |
")\n",
|
|
|
|
| 56 |
"import torch\n",
|
| 57 |
"from transformers import AutoFeatureExtractor, AutoModel, AutoTokenizer\n",
|
| 58 |
"\n",
|
| 59 |
+
"model_id = \"abr-ai/asr-19m-v2-en\"\n",
|
| 60 |
"feature_extractor = AutoFeatureExtractor.from_pretrained(\n",
|
| 61 |
" model_id, trust_remote_code=True\n",
|
| 62 |
")\n",
|