Nekodimos/Whisper-SAM

Amharic Whisper (Small) – Custom Tokenizer

Whisper-SAM (Whisper Small Amharic) is a Whisper Small model fine-tuned on ~210 hours of Amharic speech, using a custom character-level tokenizer to fully utilize the 448-token context window and eliminate byte-level inefficiencies.

---

The Problem

Whisper’s default tokenizer is byte-level, so:

• English: 1 letter ≈ 1 byte
• Amharic: 1 letter ≈ 3 bytes

That means for Amharic you end up predicting multiple byte tokens per character, hitting the decoder’s limits faster, and often getting broken byte artifacts like:

�

(the Unicode replacement character when UTF-8 decoding fails).

Also, Whisper’s decoder has a maximum text token length of ~448 tokens, so truncation happens early if every character takes 3 tokens instead of 1.

Finally, Whisper processes a fixed 30-second audio chunk as its standard spectrogram input — about 80 mel bins over those 30 seconds.

---

What Changed

• Character tokenizer: 1 Amharic character = 1 token
• You actually hit the 448 token limit in real Amharic terms
• No artificial byte inflation
• No broken � tokens
• Fewer tokens to generate per utterance

This means the decoder isn’t busy spamming byte tokens and can use its full context window (448 text-token capacity) effectively.

---

Result

• Faster convergence during training
• Faster inference (fewer tokens to predict)
• Better token efficiency
• Better accuracy for the same small model size
• Cleaner outputs with no incomplete UTF-8 artifacts

Note: Timestamp prediction is not supported in Whisper-SAM. The original Whisper timestamp tokens were not included in the custom tokenizer, so the model cannot generate or infer timestamp outputs.

---

Credit: If you use Whisper-SAM, Credit to Nekodimos would be appreciated.