alabenayed commited on
Commit
3b442c4
·
verified ·
1 Parent(s): 05baab6

update readme.md

Browse files
Files changed (1) hide show
  1. README.md +0 -26
README.md CHANGED
@@ -278,9 +278,6 @@ indicates the CPT checkpoint already carried strong dialectal representations, l
278
  conversational structure rather than the language itself. Gradient norms held at 0.59–0.68 despite
279
  the transition from quantized CPT to full-precision bf16.
280
 
281
- > **These are training-set values.** A 6,334-example held-out test split exists but has not yet
282
- > been scored, and no comparison against base `aya-expanse-8b` on the same split has been run.
283
- > Both are planned; until then, do not read 76.2% as generalisation performance.
284
 
285
  ### Downstream human evaluation
286
 
@@ -310,29 +307,6 @@ Conversational assistants, voice interfaces, and dialogue systems in Tunisian Ar
310
  particular as the generation stage downstream of a Tunisian dialect ASR front-end, which is how it
311
  was built and deployed.
312
 
313
- ## Limitations and risks
314
-
315
- - **Token accuracy is a training metric.** See the evaluation note above. Held-out numbers are not
316
- yet available.
317
- - **Regional and register skew.** The CPT corpus is dominated by written online Tunisian —
318
- social media, tweets, forums — plus ASR transcripts. Hyper-local vocabulary and interior-region
319
- variants are underrepresented, and performance on southern and rural varieties is untested.
320
- - **Augmentation-derived training data.** Roughly two-thirds of the SFT set consists of mechanical
321
- variants of ~11.9K seed pairs. This improves robustness to spelling variation but may have
322
- introduced template-like response patterns.
323
- - **Verbosity and length control.** Qualitative inspection after CPT showed long, loosely
324
- controlled outputs. SFT and the persona system prompt improved this substantially, but length
325
- calibration remains imperfect; the `max_new_tokens` and `repetition_penalty` defaults above
326
- matter.
327
- - **Arabizi input.** Training targets were Arabic script. Latin-script Arabizi input is handled
328
- poorly by the model alone — in the deployed system, a query-rewriting layer normalises Arabizi
329
- digit conventions (3→a, 7→h, 5→kh, 9→q, 8→gh) before retrieval.
330
- - **No safety evaluation.** No red-teaming, toxicity, bias, or jailbreak testing has been
331
- performed. Only 5 of the 80 base-selection prompts touched refusal behaviour. Do not deploy in a
332
- user-facing product without your own safety evaluation.
333
- - **Hallucination.** As with any 8B model. The system prompt forbids it; that is not a mechanism.
334
- Use retrieval grounding for factual queries.
335
- - **Non-commercial licence,** inherited from `aya-expanse-8b`. Research and personal use only.
336
 
337
  ---
338
 
 
278
  conversational structure rather than the language itself. Gradient norms held at 0.59–0.68 despite
279
  the transition from quantized CPT to full-precision bf16.
280
 
 
 
 
281
 
282
  ### Downstream human evaluation
283
 
 
307
  particular as the generation stage downstream of a Tunisian dialect ASR front-end, which is how it
308
  was built and deployed.
309
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
310
 
311
  ---
312