qwen3-4B-shell (RFT Adapter)

A LoRA adapter for Qwen3-4B that teaches tool-calling, bad command correction, and diagnostic thinking for shell/systems tasks.

Training Data

Synthetic dataset generated with xiaomi mimo-v2.5 — a reasoning model used to create multi-turn conversations with tool-call patterns.

Split	Samples
Train	13
Valid	3

The dataset teaches a try → fail → research → correct pattern:

User: "Count files in a directory"
Assistant: Let me try.
Assistant: [bash: ls | wc -l]
Tool: Attempted: ls | wc -l
Assistant: That wasn't right. Let me research and correct.
Assistant: [research: "count all files linux"]
Tool: Use find with -type f
Assistant: Use `find . -type f | wc -l`. `ls | wc -l` fails for files with newlines.

Behavior Trained

1. Tool calls — correct, specific commands (uv over pip, find over ls, etc.)
2. Bad command correction — recognize wrong commands, research the fix, explain why
3. Diagnostic thinking — check logs, profile processes, inspect permissions before acting
4. Safety — suggest backups, Docker/VMs for destructive operations
5. Platform awareness — pfctl on macOS, Get-Content on PowerShell, etc.

Benchmark Results

Tested on 20 cases across 4 categories:

Model	Overall	Tool Calls	Bad Fix	Diagnostic	Safety
Base	61.6	77.5	40.0	58.2	70.0
SFT	81.0	86.2	65.0	85.0	100.0
RFT	85.0	87.5	80.0	95.0	70.0
DPO	64.2	75.0	40.0	75.8	70.0

RFT scored highest overall, excelling at bad command fixes (80.0 vs SFT's 65.0) and diagnostics (95.0 vs SFT's 85.0).

Adapter Details

• Base model: unsloth/qwen3-4b-unsloth-bnb-4bit (4-bit NF4 quantized)
• LoRA rank: 16, alpha: 32, dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable params: 33M (0.81% of total)
• PEFT version: 0.19.1

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel, LoraConfig

# Load base model
bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4")
model = AutoModelForCausalLM.from_pretrained(
    "unsloth/qwen3-4b-unsloth-bnb-4bit",
    quantization_config=bnb_config,
    device_map="auto",
)

# Apply LoRA and load adapter
lora_config = LoraConfig(
    r=16, lora_alpha=32, lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    task_type="CAUSAL_LM",
)
model = PeftModel(model, lora_config)
model.load_adapter("whos-carmen/qwen3-4B-shell", adapter_name="rft")
model.set_adapter("rft")

# Chat
tokenizer = AutoTokenizer.from_pretrained("whos-carmen/qwen3-4B-shell")
messages = [
    {"role": "system", "content": "You are a shell assistant. Prefer uv over pip. Research before guessing."},
    {"role": "user", "content": "I ran `ls | wc -l` but it only counts visible files. How do I count all files?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:]))

Files

whos-carmen/qwen3-4B-shell/
├── README.md              # This file
├── rft/
│   ├── adapter_config.json
│   ├── adapter_model.safetensors
│   ├── chat_template.jinja
│   ├── tokenizer.json
│   └── tokenizer_config.json
└── data/
    ├── rft_train.jsonl    # Training data (13 samples)
    └── rft_valid.jsonl    # Validation data (3 samples)