README.md

---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen3.5-0.8B
- Qwen/Qwen3.5-2B
- Qwen/Qwen3.5-4B
library_name: gguf
tags:
- dictation
- speech-to-text
- text-cleanup
- post-asr
- qwen3.5
- gguf
- llama.cpp
- on-device
pipeline_tag: text-generation
inference: false
---

# Quill: on-device dictation cleanup models

**Quill** is a family of small language models that turn raw speech-to-text
output into clean, written text, **entirely on your own device**. It removes
filler words (*um*, *uh*, *like*, *you know*), fixes punctuation and
capitalization, repairs spoken self-corrections and false starts, and collapses
the stutters and repeats that dictation produces, without changing your words
or sending anything to the cloud.

Quill is the cleanup stage of **[Quobi](https://huggingface.co/quobi)**, a
private, offline dictation app for desktop and mobile.

## What this is

When you dictate, a speech recognizer (e.g. Whisper) produces a literal, messy
transcript:

> *"um so i was thinking like maybe we could you know meet up at three"*

Quill rewrites that into what you actually meant to write:

> **"So I was thinking maybe we could meet up at three."**

It is **not** a chatbot and not an instruction-following assistant. It does one
job: clean dictated text. Feeding it questions or commands will not get answers;
it will just clean the text.

## Base model & credit

Quill is a fine-tune of **[Qwen3.5](https://huggingface.co/Qwen)** by the Qwen
team (Alibaba), used under the **Apache 2.0** license. Qwen3.5 is a hybrid
architecture interleaving **Mamba-2 / state-space (SSM)** layers with periodic
full-attention layers, which makes the small sizes fast and memory-light,
well suited to on-device, low-latency cleanup. All credit for the base models
goes to the Qwen team; Quill only adds task-specific fine-tuning.

| Quill tier | Base model | Size (Q4_K_M) |
|---|---|---|
| `quill-0.8b-Q4_K_M.gguf` | [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) | 505 MB |
| `quill-2b-Q4_K_M.gguf` | [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) | 1.2 GB |
| `quill-4b-Q4_K_M.gguf` | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) | 2.6 GB |

## Which tier to use

| Tier | Best for | Behavior |
|---|---|---|
| **0.8B** | Phones and any CPU (recommended default) | **Verbatim**: faithful cleanup, no rephrasing |
| **2B** | Mid-range machines / a modest GPU | Verbatim + light tidying |
| **4B** | Desktops with a GPU | Verbatim + tidying + light formatting |

The smaller tiers are deliberately conservative. The **0.8B is verbatim-only by
design**: it is paired with a deterministic post-processing scaffold (symbol,
email, URL, and number normalization) so the model never has to *guess* at
conversions like "at" → `@`. This keeps the tiny model accurate and predictable;
the larger tiers take on more rewriting and structure.

## Usage (llama.cpp)

```bash
llama-server -m quill-0.8b-Q4_K_M.gguf --host 127.0.0.1 --port 8080 -ngl 99
```

**Prompt format (important).** Use ChatML with the assistant turn pre-seeded
with an **empty think block** so the model does not emit chain-of-thought:

```
<|im_start|>system
You clean up dictated text.<|im_end|>
<|im_start|>user
yeah so um the meeting is gonna be like at uh three thirty tomorrow i think<|im_end|>
<|im_start|>assistant
<think>

</think>

```

→ **"The meeting is at 3:30 tomorrow."**

> ⚠️ Do **not** pass `--jinja`. It re-enables chain-of-thought leakage. Use the
> raw prompt above (or the `/completion` endpoint) with the pre-seeded empty
> `<think></think>` block. Greedy decoding (`temperature = 0`) is recommended.

## Intended use & limitations

- **Intended:** post-ASR cleanup of first-person English dictation.
- **Not intended:** as a general assistant, translator, or summarizer; for
  languages other than English (non-English text is passed through, not
  cleaned); for safety-critical rewriting.
- Like any LM it can occasionally over- or under-edit. The verbatim tiers
  minimize this by preserving your wording; pair them with the deterministic
  scaffold for symbol/number normalization.

## License

**Apache 2.0**, inherited from the Qwen3.5 base models (also Apache 2.0). You
are free to use, modify, and redistribute, including commercially, under the
terms of the license. Fine-tuned and released as part of the **Quobi** project.