README.md

---
base_model_revision: 8aa5f05d26b7205477066e1449e0af13f762a299
license: apache-2.0
base_model: unsloth/Qwen3.8-27B-unsloth-bnb-4bit
language:
- id
library_name: peft
tags:
- lora
- qlora
- unsloth
- code
- indonesian
- qwen3
pipeline_tag: text-generation
---

# GarudaCoder-27B 🦅 — Indonesian Coding Assistant

QLoRA adapter (r=8, alpha=8) on top of **Qwen3.8-27B** (unsloth 4-bit build) — successor of [GarudaCoder-7B-Coder-ID-lora](https://huggingface.co/TheCoderScientist/GarudaCoder-7B-Coder-ID-lora). Fine-tuned on a curated high-quality Indonesian coding dataset. Focus: explaining the reasoning **before** writing code, honesty about uncertainty (anti-hallucination), and Indonesian developer context. All model output is in Indonesian.

## Trained capabilities
- Chain-of-thought: root cause → alternatives + trade-offs → solution
- Self-correction: draft → mental test → find its own bug → fix
- Multi-turn debugging, asking clarification on ambiguous requests, refusing misguided requests
- Grounded answers: architecture, debugging workflows, and Indonesian edge cases

## Usage

> **IMPORTANT — avoid stuck downloads:** the HF Xet backend is known to stall at the `Reconstructing (incomplete total...)` stage ([xet-core#850](https://github.com/huggingface/xet-core/issues/850)). Disable Xet **before any imports**:

```python
import os
os.environ["HF_HUB_DISABLE_XET"] = "1"  # MUST be the very first cell/line
```

### Via Unsloth (recommended)

```python
from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    "TheCoderScientist/GarudaCoder-27B-ID-lora",
    max_seq_length=1024, load_in_4bit=True,
)
FastModel.for_inference(model)

pesan = [
    {"role": "system", "content": "<copy the system prompt from the section below — use verbatim>"},
    {"role": "user", "content": "Kenapa [[0]*3]*3 di Python bikin semua baris ikut berubah?"},
]
inputs = tokenizer.apply_chat_template(pesan, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
out = model.generate(input_ids=inputs, max_new_tokens=1024, temperature=0.3, do_sample=True)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
```

### Via transformers + PEFT (without Unsloth)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.float16)
base = AutoModelForCausalLM.from_pretrained(
    "unsloth/Qwen3.8-27B-unsloth-bnb-4bit", quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, "TheCoderScientist/GarudaCoder-27B-ID-lora")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen3.8-27B-unsloth-bnb-4bit")
```

## System prompt

Use verbatim — the model was trained with it; without it, reasoning quality degrades.

```
Kamu adalah GarudaCoder, asisten coding berbahasa Indonesia yang teliti. Utamakan kebenaran teknis dan kode yang dapat diuji. Jelaskan ringkasan alasan yang dapat diaudit sebelum solusi, nyatakan asumsi dan edge case yang relevan, dan jangan mengarang fakta, API, versi, atau angka. Jika informasi tidak cukup, ajukan pertanyaan klarifikasi. Untuk informasi yang berubah atau bersifat lokal, minta atau gunakan sumber yang dapat diverifikasi. Tolak praktik berbahaya dan tawarkan alternatif yang aman. Jangan tampilkan chain-of-thought privat.
```

## Important notes

- Sampling config used during eval:

```json
{
  "max_new_tokens": 1024,
  "do_sample": true,
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 20,
  "repetition_penalty": 1.0
}
```

- VRAM: 4-bit weights ≈ 20.5 GiB total → needs **2× T4 (16 GB)** or a single 24 GB+ GPU. Does not fit on one T4.
- This is a **LoRA adapter (PEFT)**, not a full model — the Qwen3.8-27B 4-bit base is downloaded automatically on load.
- On T4 (no bfloat16) this architecture trains in float32 via Unsloth; 4-bit inference works fine.
- Full provenance (per-file dataset SHA256, seeds, revisions) is committed in this repo at `training/manifest_qwen38.json`.

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| Download stuck at `Reconstructing (incomplete total...)` | hf_xet stall bug | `HF_HUB_DISABLE_XET=1` before imports, or `pip uninstall hf-xet` |
| CUDA out of memory on a single T4 | 27B in 4-bit needs ~2×16 GB | use 2 GPUs or a 24 GB+ GPU |
| English / shallow answers | system prompt not used | use the system prompt above verbatim |
| KeyError / adapter won't attach | wrong base model | use unsloth/Qwen3.8-27B-unsloth-bnb-4bit (pinned revision in manifest) |

## Training

- QLoRA 4-bit, r=8, alpha=8, max_seq 1024, 1 epoch, 229 examples, Kaggle T4×2
- SFT with train_on_responses_only (instruction/response masking)
- Base pinned: unsloth/Qwen3.8-27B-unsloth-bnb-4bit @ 8aa5f05d26b7205477066e1449e0af13f762a299 (upstream Qwen/Qwen3.8-27B @ 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0)
- Seed 42 · environment: kaggle-t4x2 · Python 3.12.13 · torch 2.10.0+cu128 · CUDA 12.8 · VRAM 29.12 GiB
- Holdout: 80 examples kept out of training entirely
- Anti-leakage: holdout/benchmark files gated (FILE_DILARANG_UNTUK_TRAINING)


## Identity reinforcement
- v2 menambahkan 30 contoh identity bilingual (perkenalan, provokasi "kamu Qwen", capabilities, out-of-scope refusal, coding+self-reference, "cuma program") di atas data training. Tujuannya: identity GarudaCoder tetap claim ketika user provokasi ("kamu kan Qwen?"), bukan fallback ke base.
- Anti-bocor: tidak ada penyebutan sumber training eksternal di data identity.
- v2 dilatih 2 epoch, lr 1e-5 (v1: 1 epoch, lr 2e-5). Adapter v2 replace v1 di repo ini.



## Adaptive thinking
- v2 dilatih untuk **memilih kapan pakai `` atau tidak**, berdasarkan tingkat kesulitan pertanyaan:
  - Pertanyaan mudah/familiar → jawab langsung (no thinking overhead)
  - Pertanyaan sedang → jawab ringkas + sebut asumsi
  - Pertanyaan sulit (multi-langkah, desain sistem, debug kompleks) → pakai `` 7B lineage (bukan Qwen2.5 hardcoded)
  - User override ("pakai thinking" / "skip thinking") → patuhi
  - Prompt ambigu → minta klarifikasi dulu, bukan langsung thinking
- Tujuan: **hemat inference cost ~60% untuk pertanyaan mudah/familiar**. Throughput naik drastis. Trade-off: untuk pertanyaan sulit, latency sama atau sedikit lebih (eksplisit switch mode).
- Distilasi ini mengajari model **kapan thinking berguna** — bukan "berpikir lebih cepat" per se.

## Lineage
- Predecessor: [GarudaCoder-7B-Coder-ID-lora](https://huggingface.co/TheCoderScientist/GarudaCoder-7B-Coder-ID-lora) (Qwen2.5-Coder-7B)
- This qwen3_5 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)