Add data provenance: generated locally by Qwen3.5-27B (Apache-2.0)

7e149f2 verified 20 days ago

5.84 kB

license: apache-2.0
base_model: mlx-community/Qwen3.5-2B-MLX-4bit
library_name: mlx
tags:
  - mlx
  - lora
  - qlora
  - dictation
  - speech-to-text
  - asr-post-processing
  - text-cleanup
  - apple-silicon
pipeline_tag: text-generation
language:
  - en

SimpleWords — Dictation Cleanup (Qwen3.5-2B QLoRA adapter)

A small LoRA adapter that turns raw speech-to-text dictation into clean, publishable text — fixing punctuation, capitalization, spelling, and word-boundary errors, and removing fillers ("um", "uh"), false starts, and self-corrections — without paraphrasing or changing meaning. Built to run on-device on Apple Silicon via MLX, per-utterance, where per-call latency matters more than throughput.

This is a QLoRA adapter (4-bit base + LoRA on all linear layers) for mlx-community/Qwen3.5-2B-MLX-4bit. It is not a standalone model — load it on top of that base (see below).

Usage (MLX / mlx-lm)

A runnable version of the snippet below ships in example.py: pip install mlx-lm huggingface_hub then python example.py (or python example.py "your um raw text").

pip install mlx-lm huggingface_hub

from huggingface_hub import snapshot_download
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

# 1. download this adapter (public — no auth needed)
adapter_dir = snapshot_download("abhiram3040/simplewords-dictation-cleanup")

# 2. load the 4-bit base + this adapter
model, tok = load("mlx-community/Qwen3.5-2B-MLX-4bit", adapter_path=adapter_dir)

# 3. the FROZEN system prompt — must match training byte-for-byte, or quality regresses
SYSTEM = (
    "You clean up raw speech-to-text dictation into clear, publishable text. "
    "Fix punctuation, capitalization, spelling, and word-boundary errors, and remove "
    "fillers, false starts, and self-corrections. Do NOT paraphrase, summarize, add, "
    "or remove any meaning. If the input is already clean, return it unchanged. "
    "Output only the cleaned text, with no preamble or quotation marks."
)

def clean(raw: str) -> str:
    # message format is "inline": the system prompt is folded into the user turn
    prompt = tok.apply_chat_template(
        [{"role": "user", "content": f"{SYSTEM}\n\n{raw}"}],
        add_generation_prompt=True, tokenize=False,
    )
    out = generate(model, tok, prompt=prompt, max_tokens=256,
                   sampler=make_sampler(temp=0.0), verbose=False)  # greedy
    return out.strip()

print(clean("um so did you like uh bring any snacks wait no did you bring some snacks"))
# -> "Did you bring any snacks?"

Two things that matter for reproducing the evaluated quality:

Use the exact system prompt above (drift here is the #1 cause of format regressions).
Fold the system prompt into the user turn (inline format), and decode greedily (temp=0).

Evaluation

Fine-tuned on 21,444 raw→clean pairs (1 epoch), evaluated on a held-out 3,142-utterance test set, plus a 117-sample human review. All ship gates cleared.

Metric	Result	Gate
Character error rate (CER)	0.0142	≤ 0.03 ✅
Word error rate (WER)	0.0260	—
Punctuation F1	0.966	≥ 0.90 ✅
Casing accuracy	0.997	≥ 0.97 ✅
Hallucination rate	0.022%	≤ 0.5% ✅
Content-loss rate	0.497%	≤ 1% ✅
Format-drift rate	0.0%	≤ 0.5% ✅
Human review — critical failures	0 / 117 (0.0%)	≤ 5% ✅
Latency (warm, per utterance)	p50 363 ms · p95 1.0 s	—

CER by disfluency: clean 0.001 · light 0.015 · heavy 0.026. Hardest slice is short utterances (CER 0.029) — still under the gate.

Training

Method: QLoRA — 4-bit base + LoRA on all linear layers (rank 16, scale 2.0, dropout 0.05).
Data: 21,444 train / 2,115 val / 3,142 test, stratified by category × disfluency × length, including ~40% intentional no-op (already-clean) pairs that teach the model not to over-edit.
Recipe: batch size 16, 1 epoch (1,350 iters), cosine LR 1e-4→1e-6 (80-step warmup), completion-only loss, max_seq_length 1024, seed 42. Trained with Apple MLX (mlx_lm.lora).

Intended use & limitations

Intended: post-ASR cleanup of short English dictation utterances (notes, messages, email, reminders, etc.) in an on-device pipeline.
Out of scope: long-form documents, languages other than English, translation, summarization, or any task that changes meaning. The model is deliberately conservative — it will not rewrite content, only clean it.
Trained on synthetically generated dictation pairs; real-world ASR error distributions may differ.

License & attribution

Everything here is Apache-2.0:

This adapter — released under Apache-2.0.
Base model — Qwen/Qwen3.5-2B is Apache-2.0 (LICENSE, verified), as is the 4-bit MLX quantization mlx-community/Qwen3.5-2B-MLX-4bit.

Apache-2.0 permits commercial use, modification, and redistribution; when you redistribute, keep the license notice and this attribution. Base model © the Qwen team (Alibaba Cloud) — please cite Qwen3.5 if you build on this.

The training data is fully synthetic — every raw→clean pair was generated locally by Qwen3.5-27B (Apache-2.0) on Apple Silicon, with no cloud APIs and no third-party corpora, so the corpus itself is unencumbered. Real-world ASR error distributions may differ, so downstream users should validate on their own audio pipeline.