whisper-large-v3-ct2-float16

A CTranslate2 conversion of openai/whisper-large-v3, quantized to float16, for fast inference with faster-whisper.

Model Details

Base model: openai/whisper-large-v3
Conversion tool: CTranslate2 via ct2-transformers-converter
Inference engine: faster-whisper (SYSTRAN)
Quantization: float16
License: CC-BY-4.0

Conversion command

pip install "transformers[torch]>=4.23" ctranslate2

ct2-transformers-converter --model openai/whisper-large-v3 \
    --output_dir whisper-large-v3-ct2 \
    --copy_files tokenizer.json preprocessor_config.json \
    --quantization float16

Usage

pip install faster-whisper

from faster_whisper import WhisperModel

model = WhisperModel(
    "rlabz/whisper-large-v3-ct2-float16",  # this repo
    device="cuda",
    compute_type="float16",
)

segments, info = model.transcribe("audio.mp3")

print(f"Detected language '{info.language}' with probability {info.language_probability:.2f}")
for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

Benchmark: Speed vs. Original Whisper

Transcription speed was compared against the original openai/whisper-large-v3 model (via 🤗 transformers, float16, same GPU) on a Swahili audio sample. Full benchmark code is available in this Colab notebook.

Model	Transcription Time
Original Whisper (`transformers`, float16)	39.44s
faster-whisper (this model, CT2 float16)	4.82s
Speedup	8.18x

Both runs transcribed the same Swahili audio clip on the same GPU and precision (float16), so the difference reflects the CTranslate2 inference engine rather than a quantization or hardware advantage.

Intended Use

Drop-in replacement for openai/whisper-large-v3 wherever faster inference is needed, including transcription of Swahili and other low-resource African languages. Output quality is expected to match the original model closely, since only the inference engine and numerical precision (float16) differ — no weights were fine-tuned or altered beyond the standard CTranslate2 conversion and quantization.

Limitations

float16 quantization requires a GPU; for CPU-only environments use int8 quantization instead.
Benchmark numbers above are from a single audio sample and a single hardware configuration — actual speedup will vary with audio length, GPU, and batch settings.

License

Released under CC-BY-4.0. Attribution to OpenAI's Whisper and SYSTRAN's faster-whisper is appreciated when reusing this model.

Citation

@misc{whisper_large_v3_ct2,
  title  = {whisper-large-v3-ct2-float16},
  note   = {CTranslate2 conversion of openai/whisper-large-v3 for use with faster-whisper},
  url    = {https://github.com/SYSTRAN/faster-whisper}
}

@misc{radford2022whisper,
  title  = {Robust Speech Recognition via Large-Scale Weak Supervision},
  author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  year   = {2022},
  url    = {https://github.com/openai/whisper}
}

Downloads last month: -