README.md

---
tags:
- faster-whisper
- ctranslate2
- whisper
- quantized
- int8
- speech-recognition
- automatic-speech-recognition
- asr
- edge-ai
- low-resource
language:
- multilingual
license: apache-2.0
pipeline_tag: automatic-speech-recognition
---

# faster-whisper-small-int8

INT8-quantized [CTranslate2](https://github.com/OpenNMT/CTranslate2) export of
[`openai/whisper-small`](https://huggingface.co/openai/whisper-small), built for
[faster-whisper](https://github.com/SYSTRAN/faster-whisper) and tuned to actually run on
**low-end hardware** — tested down to **2 CPU cores / 4GB RAM**,
no GPU required.

This was built out of a real need: cloud speech-to-text pricing was too high for an
ongoing AI video generation project (text-based model, no diffusion), so this model exists
to make good speech-to-text usable on modest, everyday machines instead.

## Why this exists

- Cloud STT APIs get expensive fast at any real volume.
- The original `openai/whisper-small` checkpoint is heavier than most laptops or low-end VPS
  boxes can comfortably run.
- INT8 quantization via CTranslate2 cuts the memory/size footprint by roughly 75% with
  minimal accuracy loss, and this repo ships that export pre-built and ready to load.

## Model details

| | |
|---|---|
| Base checkpoint | [`openai/whisper-small`](https://huggingface.co/openai/whisper-small) |
| Quantization | INT8 (CTranslate2) |
| Runtime | [faster-whisper](https://github.com/SYSTRAN/faster-whisper) |
| Tested hardware floor | 2 CPU cores, 4GB RAM, no GPU |
| Size on disk | ~240 MB |
| License | Apache 2.0 (inherited from the base Whisper checkpoint) |

## Usage

```python
from faster_whisper import WhisperModel

model = WhisperModel(
    "devxyasir/faster-whisper-small-int8",
    device="cpu",
    compute_type="int8",
    cpu_threads=2,      # match your machine's core count
    num_workers=1,      # keep memory predictable on low-RAM boxes
)

segments, info = model.transcribe(
    "audio.mp3",
    beam_size=5,        # default quality; try 3 or 2 first if you need more speed
    vad_filter=True,    # skip silent stretches
)

print(f"Detected language: {info.language} ({info.language_probability:.2f})")
for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")
```

### If you need more speed than `beam_size=5` gives you

Don't jump straight to `beam_size=1` — try `3` first, it keeps most of the accuracy while
recovering a good chunk of the speed. Also worth adding:
`condition_on_previous_text=False` (faster, more stable on long/noisy audio) and a shorter
`chunk_length` (e.g. `20`) if memory is tighter than 4GB.

### Need to go below INT8?

CTranslate2 doesn't support sub-8-bit kernels on CPU. If you need to go lower than this
repo, look at a [whisper.cpp](https://github.com/ggml-org/whisper.cpp) GGML export
(Q5_0/Q4_0) of the same base checkpoint — different runtime, genuinely lower bit-width,
with a real accuracy trade-off at Q4.

## Limitations

- Quantization is applied uniformly; extremely quiet, overlapping-speaker, or heavily
  accented audio may see slightly more degradation than clean studio audio.
- Tested primarily on CPU inference; GPU users are usually better served by the original
  unquantized checkpoint or `int8_float16` on CUDA.
- Inherits any limitations and biases of the base `openai/whisper-small` Whisper checkpoint.

## Credits

This is a community-quantized export, not an official OpenAI or SYSTRAN release. If this
model is useful in your project, a credit/link back and a follow are genuinely
appreciated — it's built and maintained by one person, not a team.

---

### About the developer

**Muhammad Yasir** — Senior AI Engineer (full-stack: web, mobile, desktop, AI systems)

- 🤗 Hugging Face: [huggingface.co/devxyasir](https://huggingface.co/devxyasir)
- 💻 GitHub: [github.com/devxyasir](https://github.com/devxyasir)
- 🐦 X / Twitter: [@devxyasir](https://twitter.com/devxyasir)
- 📸 Instagram: [@devxyasir](https://instagram.com/devxyasir)
- 💼 LinkedIn: [linkedin.com/in/devxyasir](https://linkedin.com/in/devxyasir)
- 📧 Email: jamyasir0534@gmail.com
- 💬 WhatsApp only: +923156808967

**Found this useful?** A credit when you use it, and a follow on Hugging Face, genuinely
helps a solo builder keep shipping open models like this one. Open to freelance/contract
AI engineering work and full-time AI/agentic systems roles — reach out on WhatsApp or
email above.