Remove the experimental LoRA adapter: it degrades Devanagari output
Browse files- lora/README.md +0 -22
lora/README.md
DELETED
|
@@ -1,22 +0,0 @@
|
|
| 1 |
-
# Qwen3.5 0.8B multilingual LoRA
|
| 2 |
-
|
| 3 |
-
A LoRA adapter that nudges Qwen3.5 0.8B toward answering in the language it was
|
| 4 |
-
asked in, rather than drifting back to English.
|
| 5 |
-
|
| 6 |
-
- Base: `Qwen/Qwen3.5-0.8B` (the catalog row `qwen3.5-0.8b-q4_k_m`)
|
| 7 |
-
- Data: [CohereLabs/aya_dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset), Apache-2.0
|
| 8 |
-
- Rank 16, alpha 32, 1 epoch over 8,202 examples
|
| 9 |
-
- Final train loss 2.216, mean token accuracy 0.564
|
| 10 |
-
|
| 11 |
-
Aya covers 65 languages, but its bulk sits in languages Qwen's tokenizer barely
|
| 12 |
-
covers. Training on those teaches noise, so this run used 21 languages the base
|
| 13 |
-
model already handles, capped at 400 examples each so the large ones do not
|
| 14 |
-
drown the rest: Arabic, Bengali, Dutch, English, French, German, Hindi,
|
| 15 |
-
Indonesian, Italian, Japanese, Korean, Marathi, Portuguese, Russian, Simplified
|
| 16 |
-
and Traditional Chinese, Spanish, Telugu, Turkish, Urdu and Vietnamese.
|
| 17 |
-
|
| 18 |
-
Trained on an M4 Pro with PEFT in bf16, then exported with llama.cpp's
|
| 19 |
-
`convert_lora_to_gguf.py`.
|
| 20 |
-
|
| 21 |
-
One epoch on a 0.8B base is a light touch. Expect better language adherence and
|
| 22 |
-
formatting, not new knowledge.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|