README.md

metadata

pipeline_tag: automatic-speech-recognition
license: mit
base_model: facebook/s2t-medium-librispeech-asr
library_name: zeromodels
tags:
  - keras
  - zeromodels
  - speech2text
  - s2t
  - automatic-speech-recognition
  - librispeech
  - audio
  - arxiv:2010.05171
  - pytorch
  - jax
  - tf

See our collection for all versions of Speech2Text.

Run Speech2Text with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/s2t-medium-librispeech-asr

Paper: fairseq S2T: Fast Speech-to-Text Modeling with fairseq (arXiv:2010.05171) · HF Papers

Speech2Text (fairseq S2T) is a classic encoder-decoder ASR model trained on LibriSpeech. Transcripts are lowercase and unpunctuated, matching the training label style (unlike Whisper / Moonshine casing).

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of facebook/s2t-medium-librispeech-asr for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is an ASR checkpoint (Speech2TextConditionalGenerate, medium).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

import soundfile as sf
from zeromodels.models.speech2text import (
    Speech2TextProcessor,
    Speech2TextConditionalGenerate,
)

model = Speech2TextConditionalGenerate.from_weights("zeromodels/s2t-medium-librispeech-asr")
processor = Speech2TextProcessor.from_weights("zeromodels/s2t-medium-librispeech-asr")

audio, sr = sf.read("your_audio.wav", dtype="float32")  # 16 kHz mono
text = model.generate(audio, processor)
print(repr(text[0]))  # lowercase, unpunctuated LibriSpeech style

Load any Speech2Text variant the same way with from_weights("zeromodels/<variant>"):

Variant	Hub
s2t-small-librispeech-asr	zeromodels/s2t-small-librispeech-asr
s2t-medium-librispeech-asr	zeromodels/s2t-medium-librispeech-asr
s2t-large-librispeech-asr	zeromodels/s2t-large-librispeech-asr

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Prefer Speech2TextProcessor.from_weights(...) so fbank settings match.
• Pass a list of waveforms to batch (extractor pads to a common length).
• See Speech2Text docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. Speech2TextConditionalGenerate.from_weights("hf:facebook/s2t-medium-librispeech-asr").

Special Thanks

A huge thank you to the Facebook fairseq S2T authors for creating and releasing these models.

License: MIT.