WordStitch-4B v0.2 Alpha

Experimental but usable. WordStitch-4B is a 4B language model fine-tuned for Chinese-Pinyin lexical rescue. It converts Chinese Pinyin placeholders embedded in English or Chinese-English mixed text into natural English.

I forgot my yusan.
→ I forgot my umbrella.

Tomorrow wo want to take ditie to xuexiao.
→ Tomorrow I want to take the subway to school.

I want to eat hanbao.
→ I want to eat a hamburger.

WordStitch is an English writing assistant for Chinese speakers. When an English word is hard to spell, type the Pinyin of its Chinese meaning. The model performs Pinyin placeholder recovery and Pinyin typo correction using sentence context, then rewrites the complete sentence as natural English. It accepts mixed Chinese / English / Pinyin input without <pinyin> tags or special delimiters.

The intended task covers Chinese Pinyin to English conversion inside short text, Chinese-English code-mixed text, and code-switching where one or more Chinese concepts are temporarily written in Pinyin. Mildly misspelled Pinyin is supported, although recovery is not guaranteed.

More examples:

I want to take ditie to xuexiao.
→ I want to take the subway to school.

wo想take ditie去school
→ I want to take the subway to school.

Project code, training scripts, and documentation: github.com/heavry/WordStitch

Files

• repository root and adapter/: selected checkpoint-500 LoRA adapter, tokenizer, and chat template (about 142MB; the subdirectory mirror keeps downloads organized)
• merged/: merged BF16 Transformers checkpoint, split into three safetensors shards (9,098,688,065 bytes for the runtime files)
• WordStitch-4B-Q4_K_M.gguf: llama.cpp/Ollama deployment artifact, 2,708,803,936 bytes
• evaluation/: frozen inputs, outputs, reviews, summaries, and freeze manifest
• docs/: provenance, redistribution policy, and license review
• Modelfile: Ollama configuration

Which artifact should I use?

• Merged BF16 (merged/): the simplest choice for Transformers inference or further research where about 9.10GB of weight storage and suitable RAM/VRAM are available. It already includes the WordStitch LoRA changes.
• LoRA adapter (repository root or adapter/): use this for the smallest download or when you already have the exact rodrigomt training base revision. Load it with PEFT; do not apply it to a different Qwen checkpoint and assume equivalent behavior.
• Q4_K_M (WordStitch-4B-Q4_K_M.gguf): use this for llama.cpp, Ollama, CPU inference, and compact deployment. Quantization reduced the frozen WordStitch score from 89/100 to 85/100.

The merged BF16 is published with the repository metadata set to license: other. The immediate rodrigomt conversion identifies its upstream model but did not expose an explicit license field or LICENSE file when reviewed. Publication does not resolve that uncertainty or relicense the merged weights as Apache-2.0. Review NOTICE and docs/LICENSE_REVIEW.md before redistribution or commercial use.

Evaluation

The same frozen 100-example WordStitch evaluation set was used before and after LoRA training. Checkpoint 500 was selected by validation loss before the frozen set was scored.

Category	Base BF16	LoRA BF16	Mac Q4_K_M
Chinese input	10/10	9/10	9/10
Standard Pinyin rescue	5/10	10/10	9/10
Initial Pinyin	4/10	9/10	9/10
Final Pinyin	6/10	9/10	9/10
Multiple rescues	2/10	8/10	7/10
Consecutive Pinyin	1/10	6/10	5/10
Chinese/English/Pinyin	6/10	10/10	10/10
Mild Pinyin typo	5/10	10/10	9/10
Contextual ambiguity	7/10	9/10	9/10
Long mixed input	5/10	9/10	9/10
WordStitch total	51/100	89/100	85/100

This means 89/100 semantic passes on the project's 100-example frozen evaluation set; Q4_K_M scored 85/100. Evaluation was reviewed by an AI evaluator (Astra), not an independent human benchmark. The benchmark is small and is not an external certification.

General QA remained 19/20 before and after LoRA. Normal English preservation remained 20/20. These are small retention checks and do not establish broad capability preservation.

Known limitations

WordStitch-4B is experimental. It performs well on common single-token Pinyin lexical rescue, but consecutive Pinyin sequences, multiple simultaneous replacements, ambiguous transliterations, uncommon vocabulary, and quantization remain challenging. It can choose the wrong word, change correct English, or produce awkward grammar. bingxiang was observed becoming “window,” and hamburger examples sometimes omitted the article “a.” Typo performance on the frozen set must not be interpreted as universal 100% reliability.

llama.cpp

hf download heavry/WordStitch-4B \
  WordStitch-4B-Q4_K_M.gguf \
  --local-dir .

llama-cli \
  -m WordStitch-4B-Q4_K_M.gguf \
  --system-prompt "Rewrite the user input as natural, complete English. The input may mix English, Chinese, toneless Mandarin pinyin, and mildly misspelled pinyin. Recover missing English words using the sentence context. Output only the final English text, without explanations or alternatives." \
  -p "I forgot my yusan." \
  --temp 0

For an OpenAI-compatible API, including a Grok Bot VM:

hf download heavry/WordStitch-4B WordStitch-4B-Q4_K_M.gguf --local-dir ./models/wordstitch

llama-server \
  -m ./models/wordstitch/WordStitch-4B-Q4_K_M.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  -c 2048 \
  -ngl 99 \
  --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  --reasoning-budget 0

Send the WordStitch instruction as the system message and the mixed sentence as the user message. Keep temperature at 0 for concise rewriting. Put authentication in front of the server before exposing it outside a trusted host.

Ollama

hf download heavry/WordStitch-4B WordStitch-4B-Q4_K_M.gguf --local-dir .
ollama create wordstitch -f Modelfile
ollama run wordstitch "I forgot my yusan."

LoRA adapter

Load adapter/ with PEFT on the exact training base revision:

from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
from peft import PeftModel

base_id = "rodrigomt/Qwen3.5-4B-Uncensored-Aggressive"
adapter_id = "heavry/WordStitch-4B"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = Qwen3_5ForConditionalGeneration.from_pretrained(base_id, dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

Use the system instruction shown above and disable thinking in the chat template.

Merged BF16 with Transformers

Download only the merged directory:

hf download heavry/WordStitch-4B \
  --include "merged/*" \
  --local-dir ./WordStitch-4B

Load it directly without PEFT:

import torch
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration

model_dir = "./WordStitch-4B/merged"
tokenizer = AutoTokenizer.from_pretrained(model_dir)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_dir,
    dtype=torch.bfloat16,
    device_map="auto",
)

The release was verified with Transformers 5.16.1: all 723 indexed tensors loaded from the three shards, the model reported 4,539,265,536 parameters in BF16, and a real forward pass produced finite logits. Per-file SHA256 values are in merged/SHA256SUMS.

Training summary

• 9,710 training rows; 576 group-isolated validation rows
• BF16 LoRA, rank 16, alpha 32, dropout 0.05
• 30,474,240 trainable parameters
• 600 steps; checkpoint 500 selected
• best validation loss: 0.192294
• effective batch size 16; maximum length 384; learning rate 5e-5

The complete mixed-source training corpus is withheld. Published materials include aggregate manifests, generation and cleaning scripts in GitHub, attribution, and the frozen evaluation. See docs/DATA.md.

License and provenance

Project-authored code and documentation are Apache-2.0. The model provenance is Qwen/Qwen3.5-4B → HauhauCS aggressive derivative → rodrigomt safetensors conversion → WordStitch LoRA → merged BF16 / Q4_K_M. Qwen and the HauhauCS repository are marked Apache-2.0, while the immediate rodrigomt conversion lacked explicit license metadata when reviewed. For that reason, this repository remains license: other; neither the merged BF16 nor other model artifacts are presented as newly Apache-2.0-licensed. Model artifact use remains subject to upstream terms. See LICENSE, NOTICE, and docs/LICENSE_REVIEW.md. This is a provenance record, not legal advice.