Void2377 commited on
Commit
17c603c
·
verified ·
1 Parent(s): 1717bc2

Remove the experimental LoRA adapter: it degrades Devanagari output

Browse files
Files changed (1) hide show
  1. lora/README.md +0 -22
lora/README.md DELETED
@@ -1,22 +0,0 @@
1
- # Qwen3.5 0.8B multilingual LoRA
2
-
3
- A LoRA adapter that nudges Qwen3.5 0.8B toward answering in the language it was
4
- asked in, rather than drifting back to English.
5
-
6
- - Base: `Qwen/Qwen3.5-0.8B` (the catalog row `qwen3.5-0.8b-q4_k_m`)
7
- - Data: [CohereLabs/aya_dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset), Apache-2.0
8
- - Rank 16, alpha 32, 1 epoch over 8,202 examples
9
- - Final train loss 2.216, mean token accuracy 0.564
10
-
11
- Aya covers 65 languages, but its bulk sits in languages Qwen's tokenizer barely
12
- covers. Training on those teaches noise, so this run used 21 languages the base
13
- model already handles, capped at 400 examples each so the large ones do not
14
- drown the rest: Arabic, Bengali, Dutch, English, French, German, Hindi,
15
- Indonesian, Italian, Japanese, Korean, Marathi, Portuguese, Russian, Simplified
16
- and Traditional Chinese, Spanish, Telugu, Turkish, Urdu and Vietnamese.
17
-
18
- Trained on an M4 Pro with PEFT in bf16, then exported with llama.cpp's
19
- `convert_lora_to_gguf.py`.
20
-
21
- One epoch on a 0.8B base is a light touch. Expect better language adherence and
22
- formatting, not new knowledge.