Qwen-Unitopia-Style

LoRA fine-tunes of Qwen models on the UNItopia LPC mudlib, a German LPMud library (sources and documentation from ftp://unitopia.de). The goal is a model that writes LPC in the UNItopia house style: file headers, inherit lines, the room, item and monster APIs, and the German documentation format. The current release is built on Qwen3.8-27B.

Files

Everything is in qwen3.8-27b/:

File	Size	What
qwen3.8-27b-unitopia-q8_0.gguf	29.0 GB	The fine-tune. Q8_0, near-lossless, includes the multi-token-prediction block
qwen3.8-27b-unitopia-q4_k_m.gguf	16.8 GB	The fine-tune in plain Q4_K_M (no importance matrix), for smaller cards
qwen3.8-27b-base-q8_0.gguf	29.0 GB	The unmodified base model in the same Q8_0 conversion, for A/B comparison
adapter-pretrain/, adapter-sft/	435 MB each	PEFT LoRA adapters of the two phases (rank 16, alpha 32, 109M params). adapter-sft is what the merged GGUFs contain; load it on Qwen/Qwen3.8-27B with PeftModel.from_pretrained
metrics-*.jsonl, run-*.json		Training curves and resolved arguments
SHA256SUMS		Checksums of the three GGUFs

All GGUFs are text-only conversions with llama.cpp (convert_hf_to_gguf.py); the vision tower of the base checkpoint is not included. The Q8_0 files were converted directly from bf16 safetensors, the Q4_K_M with llama-quantize at default settings.

Also here: Qwen3.5-35B-A3B ("flash-lite")

qwen3.5-35b-a3b/ holds the same fine-tune applied to Qwen3.5-35B-A3B, a mixture-of-experts model with 3B active parameters: qwen3.5-35b-a3b-unitopia-q8_0.gguf (37.8 GB, Q8_0 with MTP, GGUF architecture qwen35moe), the two LoRA adapters and the training metrics. It answers at roughly the speed of a 3B model with the knowledge of a 35B one; final eval loss 1.14 against the 27B's 0.97, so the 27B writes better code and the 35B-A3B answers faster.

How it was trained

Two LoRA phases with PyTorch and PEFT on one RTX PRO 6000 (96 GB):

Phase	Data	Optimizer steps	Time	Eval loss start → end
1 causal LM	mudlib sources and docs, 4411 windows of ≤2048 tokens, 3.97M target tokens, 2 epochs	1102	274 min	1.51 → 0.97
2 SFT	2815 instruction pairs derived from the sources (write this file, explain this help page, where is X), 2 epochs	703	78 min	1.02 → 0.97

LoRA on all attention, MLP and gated-delta-net projections, bf16 base, gradient checkpointing, AdamW lr 2e-4 with warmup and cosine decay. Eval loss is measured on a held-out 5% of the same data. For scale, the same recipe gave 1.38 on Qwen3-0.6B, 1.19 on Qwen3-4B-Instruct-2507 and 1.06 on Qwen3.5-9B.

Using it

Prompts in German work best, phrased like the training data:

Implementiere `/room/kirche/treppe5.c` für die UNItopia Mudlib (Die Treppe zum Kirchturm).
Dokumentation für die Hilfeseite `rm` (UNItopia Mudlib)?
Schreibe einen einfachen NPC für die UNItopia Mudlib: ein Bäcker, der Brot verkauft und auf 'hallo' antwortet.

Training used the chat template with thinking disabled (enable_thinking=False); the model still works with thinking on. Tool calling from the base model is intact. With llama.cpp:

llama-cli -m qwen3.8-27b-unitopia-q8_0.gguf -ngl 99 -cnv

What to expect

It writes idiomatic UNItopia LPC, reproduces the documentation format, and composes new objects from the mudlib's conventions (a seller inherits /i/money/verkaeufer, an NPC /i/monster/monster with monster::create()). It is a junior builder, not a senior one: when it does not remember a function it tends to invent a plausible name instead of saying so. Give it file access to the mudlib and a system prompt that tells it where the mudlib lives and to look functions up before using them, and review what it writes before anything reaches players.

Licences

The base model is Qwen3.8-27B under its own licence. The training data is the UNItopia mudlib and documentation, which are licensed for non-commercial use only; treat these fine-tunes the same way. Base weights are redistributed here only as a conversion for comparison.