README.md

---
pipeline_tag: automatic-speech-recognition
license: mit
base_model: facebook/s2t-medium-librispeech-asr
library_name: zeromodels
tags:
- keras
- zeromodels
- speech2text
- s2t
- automatic-speech-recognition
- librispeech
- audio
- arxiv:2010.05171
- pytorch
- jax
- tf
---

## ***See [our collection](https://huggingface.co/collections/zeromodels/speech2text-6a8eaf32c86fd39eeacbb3d0) for all versions of Speech2Text.***

# Run Speech2Text with Keras 3: JAX, PyTorch, or TensorFlow

[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-Speech2Text-blue)](https://imvision12.github.io/ZeroModels/speech2text/) [![Collection](https://img.shields.io/badge/HF-Speech2Text%20collection-yellow)](https://huggingface.co/collections/zeromodels/speech2text-6a8eaf32c86fd39eeacbb3d0)

# zeromodels/s2t-medium-librispeech-asr

Paper: [fairseq S2T: Fast Speech-to-Text Modeling with fairseq (arXiv:2010.05171)](https://arxiv.org/abs/2010.05171) · [HF Papers](https://huggingface.co/papers/2010.05171)

Speech2Text (fairseq S2T) is a classic encoder-decoder ASR model trained on **LibriSpeech**. Transcripts are **lowercase and unpunctuated**, matching the training label style (unlike Whisper / Moonshine casing).

For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/s2t-medium-librispeech-asr).

Pure-**Keras 3** conversion of [`facebook/s2t-medium-librispeech-asr`](https://huggingface.co/facebook/s2t-medium-librispeech-asr) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.

This is an **ASR** checkpoint (`Speech2TextConditionalGenerate`, medium).

## ✨ Quick start

```python
import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

import soundfile as sf
from zeromodels.models.speech2text import (
    Speech2TextProcessor,
    Speech2TextConditionalGenerate,
)

model = Speech2TextConditionalGenerate.from_weights("zeromodels/s2t-medium-librispeech-asr")
processor = Speech2TextProcessor.from_weights("zeromodels/s2t-medium-librispeech-asr")

audio, sr = sf.read("your_audio.wav", dtype="float32")  # 16 kHz mono
text = model.generate(audio, processor)
print(repr(text[0]))  # lowercase, unpunctuated LibriSpeech style
```

Load any Speech2Text variant the same way with `from_weights("zeromodels/<variant>")`:

| Variant | Hub |
|---|---|
| `s2t-small-librispeech-asr` | [`zeromodels/s2t-small-librispeech-asr`](https://huggingface.co/zeromodels/s2t-small-librispeech-asr) |
| `s2t-medium-librispeech-asr` | [`zeromodels/s2t-medium-librispeech-asr`](https://huggingface.co/zeromodels/s2t-medium-librispeech-asr) |
| `s2t-large-librispeech-asr` | [`zeromodels/s2t-large-librispeech-asr`](https://huggingface.co/zeromodels/s2t-large-librispeech-asr) |

## Tips

- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Prefer `Speech2TextProcessor.from_weights(...)` so fbank settings match.
- Pass a list of waveforms to batch (extractor pads to a common length).
- See [Speech2Text docs](https://imvision12.github.io/ZeroModels/speech2text/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `Speech2TextConditionalGenerate.from_weights("hf:facebook/s2t-medium-librispeech-asr")`.

## Special Thanks

A huge thank you to the Facebook fairseq S2T authors for creating and releasing these models.

License: MIT.