How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nirajr/coedit-small-gguf:Q8_0
# Run inference directly in the terminal:
llama cli -hf nirajr/coedit-small-gguf:Q8_0
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nirajr/coedit-small-gguf:Q8_0
# Run inference directly in the terminal:
llama cli -hf nirajr/coedit-small-gguf:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf nirajr/coedit-small-gguf:Q8_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf nirajr/coedit-small-gguf:Q8_0
Use Docker
docker model run hf.co/nirajr/coedit-small-gguf:Q8_0
Quick Links

coedit-small-GGUF

Two Q8_0 GGUF files for grammar cleanup of dictated English, run with llama.cpp. Both are 77M-parameter T5 models and 83 MB on disk.

Filename What it is
coedit-small-dictation-v1-q8_0.gguf jbochi/coedit-small fine-tuned on synthetic dictation pairs. Use this one for dictation.
coedit-small-q8_0.gguf jbochi/coedit-small with unchanged weights. Kept for existing users.

The dictation model is trained for speech-to-text output: text with few or no punctuation marks, run-on sentences, filler words, and small grammar slips. It adds punctuation and sentence breaks, fixes grammar, and leaves the speaker's wording alone.

Compared with the base file, it doubles the punctuation score on TED talks, from 19.7 to 41.9. On 123 real dictation transcripts it makes 3 worse, where the base file makes 8 worse.

Usage

The prompt is Fix the grammar: followed by the text. Use greedy decoding.

llama-completion -m coedit-small-dictation-v1-q8_0.gguf --temp 0 -no-cnv \
  -p "Fix the grammar: okay so i was thinking we should move the meeting to friday because two people is on leave does that work for you"
# okay, so I was thinking we should move the meeting to Friday because two people are on leave. Does that work for you?

The same input through coedit-small-q8_0.gguf gives one sentence with no comma and no question mark:

# okay so I was thinking we should move the meeting to Friday because two people are on leave does that work for you.

Use llama-completion. llama-cli does not run this model. The context is 512 tokens.

Results

"Base" is coedit-small-q8_0.gguf. "Dictation v1" is coedit-small-dictation-v1-q8_0.gguf. Measured on 2026-11-11.

Punctuation on TED talks

The test is IWSLT 2011, a public benchmark of about 12,600 words of TED talks with all punctuation removed. The model has to put back commas, periods, and question marks.

The score is F1, the standard score for punctuation tests. It runs from 0 to 100 and combines two things: how many of the needed marks the model found, and how many of the marks it added were right. 100 means every mark found and none wrong.

Input Chunk size Base Dictation v1
Human transcript about 48 words 19.7 41.9
Human transcript about 127 words 7.8 43.4
Speech-to-text output about 48 words 17.6 38.6
Speech-to-text output about 126 words 7.5 40.0

Private dictation transcripts

123 real dictation transcripts from one speaker. A human wrote the corrected version of each one. None were used in training. The text is private and is not published.

Measure Base Dictation v1
Transcripts made worse 8 of 123, 6.5% 3 of 123, 2.4%
Share of the model's edits that were correct 54.7% 68.0%
Share of needed fixes the model made 19.8% 50.4%
GLEU 67.0 79.5
Average time per transcript 81 ms 83 ms

GLEU is a standard score for grammar correction. It compares the model's output with the human-corrected version and counts the runs of 1 to 4 words they share. It runs from 0 to 100, and 100 means the output is identical to the human version. The raw transcripts with no cleanup score 63.9.

"Made worse" was judged with the model names hidden. Times are from an M4 Mac mini with 24 GB of memory.

Essays by English learners

JFLEG is a public benchmark of 747 sentences written by people learning English, each corrected by 4 people. It is scored with GLEU.

Measure Base Dictation v1
JFLEG test GLEU 47.6 45.4

This is the one test where the dictation model scores lower, by 2.2 points out of 100. Learner essays need spelling fixes and heavy rewrites. The dictation model is trained to change as little as possible, so it leaves more of those errors in place. Use the dictation file for speech and the base file for learner writing.

Training

  • Base model: jbochi/coedit-small, which is flan-t5-small fine-tuned on Grammarly's CoEdIT dataset.
  • Data: 2,615 synthetic training pairs and 301 validation pairs, written and screened by language models. No real transcript was used. 60% of inputs have errors to fix, and 40% are clean or cut-off inputs that must come back unchanged.
  • Settings: learning rate 1e-4, 10 epochs, about 8 minutes on the Mac mini's GPU. Epoch 7 scored best on the validation pairs and was kept.

Example pairs:

Input Target
I am waiting for answer since Monday. I have been waiting for an answer since Monday.
Okay uh we start the ride at six thirty Okay, we start the ride at six thirty.
I got the job offer from the school in nakuru I got the job offer from the school in Nakuru.
Tomorrow I can't come for the ride, my knee is hurting. Tomorrow I can't come for the ride. My knee is hurting.
The pharmacist said this two medicines should not be taken together so I need to call the doctor and ask which one to stop. The pharmacist said these two medicines should not be taken together, so I need to call the doctor and ask which one to stop.
Merge the brunch after the review is done Merge the brunch after the review is done.
Okay, I'll call you after the handover. Okay, I'll call you after the handover.
Search is timing out whenever the query has more than. Search is timing out whenever the query has more than.

A misheard word such as "brunch" for "branch" is left as spoken. The model is trained to fix grammar and punctuation, never to guess the intended word.

Known limits

The dictation model is better than the base file on each of these, and still imperfect. English only.

  • Wrong edits. On the private transcripts 62 of its 194 edits were wrong, about 1 in 3. For the base file it was 43 of 95, nearly 1 in 2. Most wrong edits are punctuation.
  • Missed sentence breaks. On the TED talks it found 42.3% of the periods. The base file found 2.1%.
  • Changed words. On the TED talks it changed 3.7 words per 1,000 that needed no change, including the name "Juan" to "Julian". The base file changed 4.1 per 1,000.

If you paste the output without reading it, add your own check, such as rejecting output that drops a number or changes length by more than 30%.

Conversion notes

Both files were made with convert_hf_to_gguf.py from llama.cpp release b11149, output type Q8_0.

The script needs spiece.model, which jbochi/coedit-small does not include. The identical file from google/flan-t5-small was used.

For coedit-small-q8_0.gguf, Q8_0 output differed from F16 on 6 of 123 test sentences, all at comma level.

Credits

jbochi for the base fine-tune, Grammarly for the CoEdIT dataset, and the llama.cpp project.

Downloads last month
46
GGUF
Model size
77M params
Architecture
t5
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for nirajr/coedit-small-gguf

Finetuned
(1)
this model