How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf franktheglock/HA-2B:BF16# Run inference directly in the terminal:
llama-cli -hf franktheglock/HA-2B:BF16Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf franktheglock/HA-2B:BF16# Run inference directly in the terminal:
./llama-cli -hf franktheglock/HA-2B:BF16Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf franktheglock/HA-2B:BF16# Run inference directly in the terminal:
./build/bin/llama-cli -hf franktheglock/HA-2B:BF16Use Docker
docker model run hf.co/franktheglock/HA-2B:BF16Quick Links
HA-2B
Home Assistant Control | Fine-tuned Qwen 3.5 2B
HA-2B is a LoRA fine-tune of Qwen 3.5 2B on the Home-Assistant-Requests-V2 dataset. It's designed to understand and execute smart home control commands, turning lights on/off, managing blinds, locking doors, querying sensor states, and any other Home Assistant automation task.
Training Details
| Hyperparameter | Value |
|---|---|
| Base model | unsloth/Qwen3.5-2B |
| Dataset | acon96/Home-Assistant-Requests-V2 (208,976 rows) |
| Format | ChatML (auto-detected) |
| Sequence length | 2048 |
| Steps | 30 |
| Batch size | 2 (effective 8 with 4× gradient accumulation) |
| Learning rate | 2e-4 |
| Optimizer | AdamW 8-bit |
| LoRA rank | 16 |
| LoRA alpha | 16 |
| LoRA dropout | 0.0 |
| Target modules | Q, K, V, O, Gate, Up, Down projections |
| Precision | bfloat16 |
| Warmup steps | 5 |
| LR scheduler | Linear |
| Packing | Disabled |
| Weight decay | 0.001 |
| Gradient checkpointing | Unsloth |
| Final training loss | 1.37 |
| Hardware | Dual NVIDIA RTX 3060 12GB |
Usage
The model uses standard ChatML conversation format. Load the base model and LoRA adapter:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = "Qwen/Qwen3.5-2B"
lora_adapter = "path/to/ha-2b-adapter"
tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
base_model,
torch_dtype="bfloat16",
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, lora_adapter)
Inference Example
<|system|>
You are 'Al', a helpful AI Assistant that controls the devices in a house.
Complete the following task as instructed.
<|user|>
turn off the living room light
<|assistant|>
Turning off the living room light now.
The dataset also includes structured tool-calling examples with function definitions (HassTurnOn, HassTurnOff, etc.) for programmatic device control.
Intended Use
- Smart home voice/text assistants
- Home Assistant / openHAB / Hubitat integrations
- Tool-calling for device control
- Natural language to home automation commands
Limitations
- Fine-tuned primarily on English residential smart home scenarios
- 2B parameter size, capable but not as nuanced as larger models
- Training was brief (30 steps, ~0.14 epochs) — more steps would likely improve convergence
Hardware & Tools
- GPUs: NVIDIA RTX 3060 12GB
- Training framework: Unsloth Studio 2026.5.8
- Base model: Qwen 3.5 2B (4-bit QLoRA)
- Downloads last month
- 102
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf franktheglock/HA-2B:BF16# Run inference directly in the terminal: llama-cli -hf franktheglock/HA-2B:BF16