timothydillan/gemma4-e2b-balinese-assistant-v6

Gemma 4 E2B Balinese Assistant v6 LoRA

Experimental Balinese assistant LoRA adapter for timothydillan/gemma4-e2b-balinese-cpt.

v6 is a direct-answer follow-up to v5. v5 improved loop stability versus v4 but still failed food, Bali-description, and dialogue prompts. v6 uses a smaller, more direct SFT mix with a larger share of curated high-frequency assistant intents.

Training

• Base model: timothydillan/gemma4-e2b-balinese-cpt
• Training data: sft_train_assistant_v6.jsonl
• Rows: 2,288
• Curated direct rows: 387, AI-curated and pending native review
• Sequence length: 1,024
• Epochs: 2
• LoRA rank / alpha: 16 / 16
• Learning rate: 4e-5
• Runtime: Kaggle T4
• Final train loss: 0.4671

The v6 training run completed with finite loss. No NaN loss or non-finite training guard failure was observed.

Status

This is a research checkpoint. It should not be treated as release-quality until smoke eval and native-speaker review confirm that it improves over v5.

Known risks:

• Curated rows are AI-written and require native-speaker review.
• Low-resource Balinese generation may be grammatically or culturally wrong.
• The model may still answer off-topic or repeat phrases.
• Do not use for medical, legal, financial, safety-critical, or authoritative cultural guidance.

Loading

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "timothydillan/gemma4-e2b-balinese-assistant-v6"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(model_id, device_map="auto")

Evaluation

Primary next gate: compare against v5 with scripts/llm_assistant_smoke_eval.py, focusing on:

• no repeated Babi Guling food loop;
• direct food answer;
• direct two-person dialogue;
• no sampled geography drift on Bali prompt;
• short direct greeting behavior.