FarukSTT — Tunisian Derja Speech-to-Text

FarukSTT is the best publicly available ASR model for Tunisian Arabic (Derja), fine-tuned from Whisper Large v3. It handles real-world Tunisian speech including natural code-switching between Arabic, French, and English.

Whisper Large v3 out of the box scores above 50% WER on Tunisian Derja. This model brings that down to ~32% through targeted fine-tuning on real Tunisian conversational data.

Performance

Model	WER
Whisper Large v3 (baseline)	>50%
FarukSTT v1	33.26%
FarukSTT v2	31.99%

Evaluated on FARUKxAUTO/tunisian-asr-cleaned validation split.

Key Features

• Tunisian Derja — not MSA, real dialectal Arabic
• Code-switching — Arabic + French + English mid-sentence
• Real-world speech — podcasts, interviews, conversations
• 54k training samples — 12.7GB of audio data

Intended Use

• Tunisian Arabic transcription
• Meeting and interview transcription for Tunisian speakers
• Input for downstream NLP tasks in Derja

Limitations

• Optimized for Tunisian dialect, not Modern Standard Arabic
• WER ~32% — suitable for assisted transcription, not verbatim accuracy
• May struggle in very noisy environments

Training Data

Fine-tuned on FARUKxAUTO/tunisian-asr-cleaned — 54,156 audio-transcription pairs of real Tunisian speech including natural code-switching.

Training Procedure

v2 Hyperparameters

• learning_rate: 1e-6
• train_batch_size: 2
• gradient_accumulation_steps: 16 (effective batch: 32)
• warmup_steps: 200
• fp16: True
• optimizer: AdamW
• Framework: Transformers 4.45.0, PyTorch 2.11.0

Training Results (v1)

Step	Validation Loss	WER
500	0.3228	34.02%
2000	0.3121	32.79%
4000	0.3110	33.26%

Training Results (v2)

Step	Validation Loss	WER
250	0.3114	32.90%
500	0.3060	31.99%

Usage

from transformers import pipeline

pipe = pipeline("automatic-speech-recognition", model="FARUKxAUTO/FarukSTT")
result = pipe("audio.wav")
print(result["text"])

Citation

If you use this model, please credit:

FarukSTT by FARUK BATTIKH — Tunisian Derja ASR, 2024