|
|
| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - home-assistant |
| - smart-home |
| - qwen |
| - unsloth |
| - lora |
| - fine-tuning |
| base_model: unsloth/Qwen3.5-2B |
| datasets: |
| - acon96/Home-Assistant-Requests-V2 |
| --- |
| |
| # HA-2B |
|
|
| **Home Assistant Control | Fine-tuned Qwen 3.5 2B** |
|
|
| HA-2B is a LoRA fine-tune of [Qwen 3.5 2B](https://huggingface.co/Qwen/Qwen3.5-2B) on the [Home-Assistant-Requests-V2](https://huggingface.co/datasets/acon96/Home-Assistant-Requests-V2) dataset. It's designed to understand and execute smart home control commands, turning lights on/off, managing blinds, locking doors, querying sensor states, and any other Home Assistant automation task. |
|
|
| ## Training Details |
|
|
| | Hyperparameter | Value | |
| |---|---| |
| | Base model | `unsloth/Qwen3.5-2B` | |
| | Dataset | `acon96/Home-Assistant-Requests-V2` (208,976 rows) | |
| | Format | ChatML (auto-detected) | |
| | Sequence length | 2048 | |
| | Steps | 30 | |
| | Batch size | 2 (effective 8 with 4× gradient accumulation) | |
| | Learning rate | 2e-4 | |
| | Optimizer | AdamW 8-bit | |
| | LoRA rank | 16 | |
| | LoRA alpha | 16 | |
| | LoRA dropout | 0.0 | |
| | Target modules | Q, K, V, O, Gate, Up, Down projections | |
| | Precision | bfloat16 | |
| | Warmup steps | 5 | |
| | LR scheduler | Linear | |
| | Packing | Disabled | |
| | Weight decay | 0.001 | |
| | Gradient checkpointing | Unsloth | |
| | Final training loss | 1.37 | |
| | Hardware | Dual NVIDIA RTX 3060 12GB | |
|
|
|
|
| ## Usage |
|
|
| The model uses standard ChatML conversation format. Load the base model and LoRA adapter: |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| base_model = "Qwen/Qwen3.5-2B" |
| lora_adapter = "path/to/ha-2b-adapter" |
| |
| tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True) |
| model = AutoModelForCausalLM.from_pretrained( |
| base_model, |
| torch_dtype="bfloat16", |
| device_map="auto", |
| trust_remote_code=True, |
| ) |
| model = PeftModel.from_pretrained(model, lora_adapter) |
| ``` |
|
|
| ### Inference Example |
|
|
| ``` |
| <|system|> |
| You are 'Al', a helpful AI Assistant that controls the devices in a house. |
| Complete the following task as instructed. |
| <|user|> |
| turn off the living room light |
| <|assistant|> |
| Turning off the living room light now. |
| ``` |
|
|
| The dataset also includes structured tool-calling examples with function definitions (`HassTurnOn`, `HassTurnOff`, etc.) for programmatic device control. |
|
|
| ## Intended Use |
|
|
| - Smart home voice/text assistants |
| - Home Assistant / openHAB / Hubitat integrations |
| - Tool-calling for device control |
| - Natural language to home automation commands |
|
|
| ## Limitations |
|
|
| - Fine-tuned primarily on English residential smart home scenarios |
| - 2B parameter size, capable but not as nuanced as larger models |
| - Training was brief (30 steps, ~0.14 epochs) — more steps would likely improve convergence |
|
|
|
|
| ## Hardware & Tools |
|
|
| - **GPUs:** NVIDIA RTX 3060 12GB |
| - **Training framework:** [Unsloth](https://github.com/unslothai/unsloth) Studio 2026.5.8 |
| - **Base model:** Qwen 3.5 2B (4-bit QLoRA) |
|
|
