README.md

---
license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
library_name: transformers
pipeline_tag: text-generation
language:
  - en
  - es
tags:
  - qwen2
  - code
  - kotlin
  - android
  - jetpack-compose
  - unsloth
  - lora
  - gguf
datasets:
  - giggiovpg/ornith-android-instruct
  - giggiovpg/android-kotlin-compose-compiler-verified
  - microsoft/NextCoderDataset
  - glaiveai/glaive-code-assistant-v3
  - theblackcat102/evol-codealpaca-v1
  - sahil2801/CodeAlpaca-20k
  - genqa/GenQA
---

# LaboAI-0.4.0-3B

LaboAI-0.4.0-3B is a 3B-parameter code model fine-tuned for **Android / Kotlin / Jetpack Compose** development. It is built on top of [Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct) using QLoRA with [Unsloth](https://github.com/unslothai/unsloth), and the LoRA weights are merged into the released model.

## Model details

| | |
|---|---|
| **Developer** | LaboAI |
| **Base model** | Qwen/Qwen2.5-Coder-3B-Instruct |
| **Parameters** | ~3B |
| **Fine-tuning method** | QLoRA (4-bit base, LoRA r=32, alpha=64) |
| **Context used in training** | 1024 tokens |
| **Languages** | Kotlin, Java, general code; English and Spanish prompts |
| **Available formats** | Merged safetensors, GGUF `q4_k_m` |

## Intended use

- Generating and explaining Kotlin code for Android apps
- Jetpack Compose UI snippets, ViewModels, state handling, Room, Retrofit, coroutines
- Fixing and refactoring small to medium Kotlin snippets
- Local inference on modest hardware via the GGUF build (llama.cpp, Ollama, LM Studio)

## Prompt format

The model was trained with a plain instruction/response template (not the Qwen chat template), so for best results use:

```
### Instruction:
{your request}

### Response:
```

## How to use

### Transformers

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

repo = "LaboAI/LaboAI-0.4.0-3B"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype=torch.float16, device_map="auto"
)

prompt = (
    "### Instruction:\n"
    "Write a Jetpack Compose screen with a counter and a button to increment it.\n\n"
    "### Response:\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```

### llama.cpp / GGUF

```bash
llama-cli -m LaboAI-0.4.0-3B-Q4_K_M.gguf -c 2048 --temp 0.2 \
  -p "### Instruction:\nCreate a Room DAO for a Note entity.\n\n### Response:\n"
```

## Training data

The training mix was filtered for Android/Kotlin/Jetpack content (keyword filter on the text fields) and formatted into the instruction/response template above:

| Dataset | Notes |
|---|---|
| giggiovpg/ornith-android-instruct | Android instruct data |
| giggiovpg/android-kotlin-compose-compiler-verified | Compose code verified by compiler |
| microsoft/NextCoderDataset | Kotlin subset, capped at 10,000 |
| glaiveai/glaive-code-assistant-v3 | Android/Kotlin subset, capped at 6,000 |
| theblackcat102/evol-codealpaca-v1 | Android/Kotlin subset |
| sahil2801/CodeAlpaca-20k | Android/Kotlin subset |
| genqa/GenQA (code split) | Android/Kotlin subset, capped at 4,000 |

Examples with empty or near-empty text were removed, and the final dataset was shuffled (seed 42).

## Training procedure

| Hyperparameter | Value |
|---|---|
| LoRA rank / alpha / dropout | 32 / 64 / 0 |
| Target modules | q, k, v, o, gate, up, down projections |
| Max sequence length | 1024 |
| Per-device batch size | 2 |
| Gradient accumulation | 4 (effective batch 8) |
| Steps | 4,500 |
| Warmup steps | 225 |
| Learning rate | 1e-4 (cosine) |
| Weight decay | 0.01 |
| Optimizer | AdamW 8-bit |
| Precision | fp16 (T4) |
| Hardware | 1x NVIDIA T4 (Kaggle) |
| Framework | Unsloth + TRL SFTTrainer |

## Limitations and risks

- It is a small 3B model: it can produce code that does not compile, uses deprecated or non-existent APIs, or contains subtle bugs. Always review and test the output.
- Trained with a 1024-token context, so very long files or multi-file projects may degrade quality.
- The keyword-based filtering may have let in some off-topic examples.
- Android and Compose APIs change quickly; the model may not know the latest library versions.
- Not evaluated on formal benchmarks yet; no quantitative results are claimed.

## License

This model is derived from Qwen2.5-Coder-3B-Instruct and inherits its license (Qwen Research License). Please check the base model's license for terms of use, especially for commercial use. The training datasets have their own licenses, which you should also review.

## Acknowledgements

- [Qwen team](https://huggingface.co/Qwen) for the base model
- [Unsloth](https://github.com/unslothai/unsloth) for efficient fine-tuning
- The authors of the datasets listed above

## Citation

```bibtex
@misc{laboai2026,
  title  = {LaboAI-0.4.0-3B: a Kotlin/Android code model},
  author = {LaboAI},
  year   = {2026},
  url    = {https://huggingface.co/LaboAI/LaboAI-0.4.0-3B}
}
```