Text Generation
PEFT
Safetensors
Tunisian Arabic
Arabic
tunisian-arabic
derja
darija
dialectal-arabic
low-resource
arabic
qlora
continued-pretraining
sft
conversational
Instructions to use alabenayed/TounsiLM-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use alabenayed/TounsiLM-8b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/aya-expanse-8b") model = PeftModel.from_pretrained(base_model, "alabenayed/TounsiLM-8b") - Notebooks
- Google Colab
- Kaggle
update readme.md
Browse files
README.md
CHANGED
|
@@ -278,9 +278,6 @@ indicates the CPT checkpoint already carried strong dialectal representations, l
|
|
| 278 |
conversational structure rather than the language itself. Gradient norms held at 0.59–0.68 despite
|
| 279 |
the transition from quantized CPT to full-precision bf16.
|
| 280 |
|
| 281 |
-
> **These are training-set values.** A 6,334-example held-out test split exists but has not yet
|
| 282 |
-
> been scored, and no comparison against base `aya-expanse-8b` on the same split has been run.
|
| 283 |
-
> Both are planned; until then, do not read 76.2% as generalisation performance.
|
| 284 |
|
| 285 |
### Downstream human evaluation
|
| 286 |
|
|
@@ -310,29 +307,6 @@ Conversational assistants, voice interfaces, and dialogue systems in Tunisian Ar
|
|
| 310 |
particular as the generation stage downstream of a Tunisian dialect ASR front-end, which is how it
|
| 311 |
was built and deployed.
|
| 312 |
|
| 313 |
-
## Limitations and risks
|
| 314 |
-
|
| 315 |
-
- **Token accuracy is a training metric.** See the evaluation note above. Held-out numbers are not
|
| 316 |
-
yet available.
|
| 317 |
-
- **Regional and register skew.** The CPT corpus is dominated by written online Tunisian —
|
| 318 |
-
social media, tweets, forums — plus ASR transcripts. Hyper-local vocabulary and interior-region
|
| 319 |
-
variants are underrepresented, and performance on southern and rural varieties is untested.
|
| 320 |
-
- **Augmentation-derived training data.** Roughly two-thirds of the SFT set consists of mechanical
|
| 321 |
-
variants of ~11.9K seed pairs. This improves robustness to spelling variation but may have
|
| 322 |
-
introduced template-like response patterns.
|
| 323 |
-
- **Verbosity and length control.** Qualitative inspection after CPT showed long, loosely
|
| 324 |
-
controlled outputs. SFT and the persona system prompt improved this substantially, but length
|
| 325 |
-
calibration remains imperfect; the `max_new_tokens` and `repetition_penalty` defaults above
|
| 326 |
-
matter.
|
| 327 |
-
- **Arabizi input.** Training targets were Arabic script. Latin-script Arabizi input is handled
|
| 328 |
-
poorly by the model alone — in the deployed system, a query-rewriting layer normalises Arabizi
|
| 329 |
-
digit conventions (3→a, 7→h, 5→kh, 9→q, 8→gh) before retrieval.
|
| 330 |
-
- **No safety evaluation.** No red-teaming, toxicity, bias, or jailbreak testing has been
|
| 331 |
-
performed. Only 5 of the 80 base-selection prompts touched refusal behaviour. Do not deploy in a
|
| 332 |
-
user-facing product without your own safety evaluation.
|
| 333 |
-
- **Hallucination.** As with any 8B model. The system prompt forbids it; that is not a mechanism.
|
| 334 |
-
Use retrieval grounding for factual queries.
|
| 335 |
-
- **Non-commercial licence,** inherited from `aya-expanse-8b`. Research and personal use only.
|
| 336 |
|
| 337 |
---
|
| 338 |
|
|
|
|
| 278 |
conversational structure rather than the language itself. Gradient norms held at 0.59–0.68 despite
|
| 279 |
the transition from quantized CPT to full-precision bf16.
|
| 280 |
|
|
|
|
|
|
|
|
|
|
| 281 |
|
| 282 |
### Downstream human evaluation
|
| 283 |
|
|
|
|
| 307 |
particular as the generation stage downstream of a Tunisian dialect ASR front-end, which is how it
|
| 308 |
was built and deployed.
|
| 309 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 310 |
|
| 311 |
---
|
| 312 |
|