leafstudiosDot/incotel-hanny

Hanny 4B chat adapter (M5 SFT)

A LoRA adapter for Qwen/Qwen3.5-4B, trained through that model's own chat template with loss on assistant tokens only. It is a public experiment, not a general-purpose assistant.

What this is

• Base model: Qwen/Qwen3.5-4B
• Adapter weights sha256[:16]: de560f905163061b (85.0 MB)
• LoRA r=16, alpha=32, targets: down_proj, v_proj, gate_proj, q_proj, o_proj, up_proj, k_proj
• Run: m5-4b-ddp | precision: 4-bit nf4 | chat template: a4aee8afcf2e0711
• Optimizer steps: 250 (best at 250) | tokens processed: 871,734
• Distributed: DDP across 2 GPUs (torch.distributed.run); the effective batch is the per-GPU batch times 2.

Training data

source	split	rows	licence
allenai/tulu-3-sft-mixture	train	40000	odc-by
HuggingFaceH4/ultrachat_200k	train_sft	40000	mit

Rows read: 80000; conversations kept: 79999; training examples: 161208; duplicates removed: 1.

One example is produced per assistant turn, so the context is repeated across turns of the same conversation.

Evaluation

Held-out assistant-token loss on held-out conversations, measured before and after training in the same process at the same precision:

	loss
base	1.4139 +- 0.0671
adapter (best)	1.0698
improvement (before - after)	0.3442 (paired se 0.0309)

Both numbers come from the same run at the same precision, and the difference is paired item-by-item, so it is the tighter comparison. This is not comparable with any fp16 number in the producing repository; 4-bit quantisation itself costs ~0.09 nats.

The M5 gate (chateval.py) scored 50 chat pairs (10 multi-turn) and reported a paired chat-loss change of +0.0574 +- 0.0204 (positive = the adapter's loss is lower). The gate process completed (exit 0). That probe set is a floor, not a certification: it can catch a collapse, not establish a general assistant. Refusal p-values: {"harmful": 1.0, "benign": 1.0}.

Code ability was measured by execution, and it did not change measurably. On 44 small tasks the base model passed 36/44 and this adapter 35/44 (McNemar exact p = 1.0000; 4 base-only, 3 adapter-only). 2 base and 1 adapter replies hit the 256-token cap, so both rates are floors. This is a task-level execution check, not a coding-agent benchmark: it cannot see tool use, multi-file edits, or whether the code is idiomatic.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained('Qwen/Qwen3.5-4B')
base = AutoModelForCausalLM.from_pretrained('Qwen/Qwen3.5-4B', dtype='float16', device_map='auto')
model = PeftModel.from_pretrained(base, 'YOUR-NAME/adapter-repo')
model.eval()

messages = [{'role': 'user', 'content': 'Hello'}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True,
                              return_tensors='pt', enable_thinking=False)
out = model.generate(ids, max_new_tokens=256, do_sample=True,
                     temperature=0.7, top_p=0.9)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Limitations

• Qwen/Qwen3.5-4B is a small model. Fine-tuning changes behaviour; it does not add knowledge the base model lacks.
• This is not a ChatGPT- or Claude-class assistant, and no amount of further LoRA training on this base will make it one.
• Refusal behaviour is not guaranteed. The gate scores 50 chat pairs (10 multi-turn) and 26 safety probes; that is a floor, not a safety audit.
• Generation settings used during development: temperature 0.7, top_p 0.9, top_k 20, thinking off. Different settings are a different model behaviour.

Provenance and rights

• Provenance class: public-mix. Trained only on Hub datasets, listed in the card. Check each dataset's licence before publishing.
• The publisher states: Trained only on allenai/tulu-3-sft-mixture (ODC-BY) and HuggingFaceH4/ultrachat_200k (MIT); no private code or user data was used. The publisher does not relicense the underlying datasets; attribution is in this card.