MigsN9/LoRA-Qwen3.5-4B-tool-usage

Qwen3.5-4B — Tool Dispatcher LoRA

LoRA adapter fine-tuned on top of Qwen/Qwen3.5-4B for use as a lightweight on-device tool dispatcher in a personal phone assistant.

What it does

Given a user utterance, the model outputs a structured JSON tool dispatch decision:

{"tool": "calendar", "function": "create", "content": "meeting with Bob", "params": {"time": "Friday 3pm", "contact": "Bob"}}

Field	Type	Meaning
tool	str	Target tool name
function	str	Specific action for the tool
content	str | null	Normalized user request string
params	obj | null	Action-specific parameters

Supported tools

Tool	Functions
calendar	view, create, edit, delete
phone	call, text
whatsapp	send
email	compose
navigate	directions
maps	search
camera	photo
music	play, pause, resume, skip, previous
volume	set
alarm	set, cancel
timer	set, cancel
reminder	set
wifi	toggle
bluetooth	toggle
flashlight	toggle
app	open
notes	create, search, open, share
weather	current, forecast
search	web, nearby
research	arxiv
noop	noop
notool	notool

Training details

Param	Value
Base model	Qwen/Qwen3.5-4B
LoRA r	16
LoRA alpha	16
LoRA dropout	0.05
Epochs	3
Batch size	4 (grad accum ×4 = effective 16)
Learning rate	2e-4
Max seq len	512
Quantization	q8_0 GGUF

Training data generated via teacher distillation (Qwen3.5-9B+ served via llama-cpp-python), covering diverse utterance styles: personal assistant, command, and casual speech.

System prompt

<TOOL>You are a tool dispatcher. For each user message, output ONLY a JSON object.
Schema: {"tool": str, "function": str, "content": str|null, "params": obj|null}
- tool: tool name (calendar, phone, whatsapp, email, navigate, maps, camera, music, volume, alarm, timer, reminder, wifi, bluetooth, flashlight, web, app, notes, noop, weather, search, research)
- function: specific action for the tool
- content: normalized user request string, or null if not applicable
- params: object with action-specific fields (contact, time, target, state, level, to, subject, delay, snaps, label, title, days, radius, author), or null
Output ONLY valid JSON. No explanation.

Usage

Served via llama-cpp-python on-device:

llama-server -m qwen3.5-4b-tool-usage-q8_0.gguf -c 2048 -ngl 99 --port 8081

import requests, json

resp = requests.post("http://localhost:8081/v1/chat/completions", json={
    "messages": [
        {"role": "system", "content": "<TOOL>..."},
        {"role": "user",   "content": "remind me to call mom tomorrow at 9am"}
    ],
    "max_tokens": 128,
    "temperature": 0.0
})
print(json.loads(resp.json()["choices"][0]["message"]["content"]))
# {"tool": "reminder", "function": "set", "content": "call mom", "params": {"time": "tomorrow 9am", "contact": "mom"}}

Repo structure

finetune_module/
├── distill/        # Teacher distillation pipeline
├── finetune/       # LoRA training, merge, GGUF conversion, upload
├── test/           # Eval harness
├── models/         # GGUF inference files
└── output/         # Generated JSONL datasets