Automatic Speech Recognition
Transformers
Safetensors
fun_asr_nano
text-generation
speech-recognition
asr
end-to-end
multilingual
streaming
arxiv:2407.04051
Instructions to use FunAudioLLM/Fun-ASR-Nano-2512-hf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FunAudioLLM/Fun-ASR-Nano-2512-hf with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="FunAudioLLM/Fun-ASR-Nano-2512-hf")# Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("FunAudioLLM/Fun-ASR-Nano-2512-hf", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update Transformers quickstart for PR 46180
Browse filesPin the verified rebased PR head, document structured language/context/hotword inputs, and remove the obsolete serialized default prompt field.
- README.md +20 -4
- processor_config.json +0 -1
README.md
CHANGED
|
@@ -40,7 +40,7 @@ With [PyTorch installed for your platform](https://pytorch.org/get-started/local
|
|
| 40 |
|
| 41 |
```bash
|
| 42 |
python -m pip install accelerate librosa \
|
| 43 |
-
"transformers @ https://github.com/huggingface/transformers/archive/
|
| 44 |
```
|
| 45 |
|
| 46 |
```python
|
|
@@ -56,11 +56,15 @@ model = AutoModelForSpeechSeq2Seq.from_pretrained(
|
|
| 56 |
)
|
| 57 |
|
| 58 |
audio_url = "https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/resolve/main/example/en.mp3"
|
| 59 |
-
inputs = processor.apply_transcription_request(
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
generated_ids = model.generate(**inputs, max_new_tokens=200)
|
| 62 |
generated_ids = generated_ids[:, inputs.input_ids.shape[1] :]
|
| 63 |
-
print(processor.
|
| 64 |
```
|
| 65 |
|
| 66 |
Expected transcription:
|
|
@@ -69,4 +73,16 @@ Expected transcription:
|
|
| 69 |
The tribal chieftain called for the boy, and presented him with fifty pieces of gold.
|
| 70 |
```
|
| 71 |
|
| 72 |
-
For
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
```bash
|
| 42 |
python -m pip install accelerate librosa \
|
| 43 |
+
"transformers @ https://github.com/huggingface/transformers/archive/4e199f7990ab38838210fc02726f34f6f25d0097.zip"
|
| 44 |
```
|
| 45 |
|
| 46 |
```python
|
|
|
|
| 56 |
)
|
| 57 |
|
| 58 |
audio_url = "https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/resolve/main/example/en.mp3"
|
| 59 |
+
inputs = processor.apply_transcription_request(
|
| 60 |
+
audio=audio_url,
|
| 61 |
+
language="en",
|
| 62 |
+
return_tensors="pt",
|
| 63 |
+
).to(model.device)
|
| 64 |
|
| 65 |
generated_ids = model.generate(**inputs, max_new_tokens=200)
|
| 66 |
generated_ids = generated_ids[:, inputs.input_ids.shape[1] :]
|
| 67 |
+
print(processor.decode(generated_ids, skip_special_tokens=True)[0])
|
| 68 |
```
|
| 69 |
|
| 70 |
Expected transcription:
|
|
|
|
| 73 |
The tribal chieftain called for the boy, and presented him with fifty pieces of gold.
|
| 74 |
```
|
| 75 |
|
| 76 |
+
For domain-specific transcription, pass contextual information with `prompt` and hotwords with `keywords`; the checkpoint chat template builds the complete instruction:
|
| 77 |
+
|
| 78 |
+
```python
|
| 79 |
+
inputs = processor.apply_transcription_request(
|
| 80 |
+
audio=audio_url,
|
| 81 |
+
language="en",
|
| 82 |
+
prompt="A tribal story involving a chieftain and a boy.",
|
| 83 |
+
keywords=["tribal chieftain", "fifty pieces of gold"],
|
| 84 |
+
return_tensors="pt",
|
| 85 |
+
).to(model.device)
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
`language` accepts Chinese, English, and Japanese as ISO codes, full English names, or native names. For batch inference, training, `torch.compile`, benchmarks, and the full model description, see the [Transformers documentation](https://github.com/huggingface/transformers/blob/4e199f7990ab38838210fc02726f34f6f25d0097/docs/source/en/model_doc/fun_asr_nano.md) and the [original model card](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512).
|
processor_config.json
CHANGED
|
@@ -1,6 +1,5 @@
|
|
| 1 |
{
|
| 2 |
"audio_token": "<|object_ref_start|>",
|
| 3 |
-
"default_transcription_prompt": "Transcribe the audio:",
|
| 4 |
"feature_extractor": {
|
| 5 |
"feature_extractor_type": "FunAsrNanoFeatureExtractor",
|
| 6 |
"feature_size": 80,
|
|
|
|
| 1 |
{
|
| 2 |
"audio_token": "<|object_ref_start|>",
|
|
|
|
| 3 |
"feature_extractor": {
|
| 4 |
"feature_extractor_type": "FunAsrNanoFeatureExtractor",
|
| 5 |
"feature_size": 80,
|