Base model  |  Technical report  |  GitHub  |  License  |  Contact

---

KeelCore-750M-Grammar

KeelCore-750M-Grammar is a LoRA adapter for KeelCore-750M that corrects spelling, grammar and punctuation. It fixes the mistakes and keeps the wording, tone and meaning, so the result reads like the original with the errors removed. It is a 73 MB file, not a full model: it needs the frozen base model to run.

• Task: correcting English spelling, grammar, capitalization and punctuation (messages, emails, notes, short documents)
• Size: rank-16 LoRA, 18.29M trainable parameters (2.43% of the base)
• Runs: on CPU or GPU with PyTorch, or in a browser through ONNX Runtime Web
• Terms of use: KeelCore License, Version 1.0 (see License)
• Author: Saravanan A R

Model Information

Description

You give the adapter a piece of text and it returns the same text with the errors fixed. It is a corrector, not a rewriter: on the held-out set, outputs are within 1% of the input's length on average, and the character-level match with the input is 94 chrF. The base model is an encoder–decoder, so it reads the whole input first and can copy names, URLs and other exact text straight into the output. The adapter teaches it the correction task. It is not a chat model and it does not answer questions or follow instructions in the text. It only corrects the text it is given.

Examples

These come from the final adapter, run with the default settings described under Usage. The inputs are not from the training data (see Hand-written checks).

Input	Output
She dont like the new office because it are too noisy.	She doesn't like the new office because it is too noisy.
i have went to the store but they was closed.	I have gone to the store, but they were closed.
He have two brother and a sister who lives in canada.	He has two brothers and a sister who lives in Canada.
If I would have known, I would of called you earlier.	If I had known, I would have called you earlier.
Please send me the informations about the meeting, I want to know whos coming.	Please send me the information about the meeting. I want to know who's coming.
i recieve your mesage yesterday and will definately reply tommorow	I received your message yesterday and will definitely reply tomorrow.
Visit https://example.com/docs/start for the setup guide it explains everything	Visit https://example.com/docs/start for the setup guide; it explains everything.
Contact support@example.com or call 1-800-555-0199 if the error persist after restarting the device	Contact support@example.com or call 1-800-555-0199 if the error persists after restarting the device.

Inputs and outputs

Direction	Description
Input	Plain English text. The code adds the frame <style:grammar> <text_to_rewrite> … </text_to_rewrite> </s> for you.
Output	The rewritten text, up to 384 new tokens with the default settings.

Intended use

• Proofreading drafts: chat messages, emails, tickets, notes and short documents.
• A correction step inside an application, on device or in a browser, where the text should not leave the user's machine.
• A starting point for researchers who study small adapters on a shared base model.

Out of scope. Text where every character must survive unchanged (code, configuration, version numbers, financial figures, quotations). Text that is already correct: the adapter was trained almost only on examples with something to fix, so it tends to edit anyway (see Limitations). Heavily garbled text, where it can introduce new mistakes. Languages other than English. Changing style or tone: it does not make text more formal, shorter or longer.

Usage

The adapter file is in a KeelCore format (adapter.safetensors with adapter_config.json), not the PEFT format, and it is loaded by the code in the KeelCore GitHub repository. It checks a fingerprint of the base weights and refuses to load onto any base other than KeelCore-750M.

git clone https://github.com/saravanan-a-r/KeelCore
pip install torch safetensors sentencepiece pyyaml huggingface_hub

import sys
from huggingface_hub import snapshot_download

sys.path.insert(0, "KeelCore/SFT_adapter_rephrase")
from src.inference import RephraseAdapter

# The base needs only these four files; the ONNX folder is not required.
base = snapshot_download(
    "KeelCore/KeelCore-750M",
    allow_patterns=["config.json", "model.safetensors", "spm.model", "token_map.json"],
)
adapter = snapshot_download("KeelCore/KeelCore-750M-Grammar", ignore_patterns=["onnx/*"])  # PyTorch files only

rephraser = RephraseAdapter.load(adapter, base_checkpoint=base, tokenizer_dir=base, device="cpu")  # or "cuda"
print(rephraser.rephrase(["me and him goes to the market yesterday."]))

rephrase() takes a list of texts and returns a list of rewrites. The base loads in float32 (about 3 GB of memory).

Decoding. The defaults are greedy decoding, a ban on repeating any 3-token sequence, and at most 384 new tokens. The tokenizer writes every digit as its own token, so the ban would force numbers to change (50000 becomes 5000). The default therefore exempts the ten digit tokens from the ban. If you write your own decoding loop, keep that exemption.

Download only what you need. Both formats are in this repository, in separate places, so you never have to fetch both. For PyTorch, use ignore_patterns=["onnx/*"] as above (73 MB). For the browser or ONNX Runtime, use snapshot_download("KeelCore/KeelCore-750M-Grammar", allow_patterns=["onnx/*"]) (73 MB), or request the two .onnx_adapter files directly by URL. The base model follows the same rule: its ONNX export lives in onnx/int8/ and is not downloaded by the PyTorch snippet.

In the browser. onnx/ holds this adapter converted to ONNX Runtime adapter files. They load on the int8 ONNX export in the base repository (onnx/int8/) through ONNX Runtime's LoraAdapter support (activeLoraAdapters run option). The base graphs, tokenizer and model.json are in onnx/int8/ of the base repository; onnx/adapter.json here lists the two adapter files with their hashes and the base they fit. The EchoMeBetter extension in the GitHub repository is a working example.

Repository contents

File	Size	Description
adapter.safetensors	73 MB	LoRA weights (fp32) and the learned embeddings of the three frame tokens
adapter_config.json	2 KB	LoRA settings, frame tokens and the fingerprint of the base it needs
config.json	1 KB	Names the base model and the style. The Hub counts a download of this repository when this file is requested
onnx/encoder.onnx_adapter, onnx/decoder.onnx_adapter	37 MB each	The same adapter for the ONNX export of the base (encoder and decoder graphs)
onnx/adapter.json	1 KB	The adapter's manifest for ONNX runners: style token, the two files with sizes and SHA-256 hashes, and the base it needs
training_curves.png, banner.png		Images used on this page
LICENSE		KeelCore License, Version 1.0

Adapter details

Base model	KeelCore-750M, pretraining step 90,000 (weights unchanged)
Method	LoRA with rank-stabilized scaling, rank 16, alpha 8 (scale 2.0), dropout 0.05
Where	Every linear layer of the encoder and decoder: self-attention, cross-attention (q, k, v, o) and feed-forward (gate, up, down), 432 LoRA pairs
Also trained	Embeddings of the three frame tokens <style:grammar>, <text_to_rewrite>, </text_to_rewrite>
Trainable parameters	18,287,616

Training

The adapter was trained on about 1.49 million correction examples for 2 passes (92,990 optimizer steps), with 20,000 further examples held out for evaluation. The base weights stayed frozen. Almost every training target differs from its input: only 239 of the 1.49 million (0.02%) are unchanged, so the adapter saw almost no "nothing to fix" examples.

Setting	Value
Effective batch size	32 examples
Learning rate	3e-4, 100 warm-up steps, cosine decay to 3e-5
Optimizer	AdamW, no weight decay, gradient clipping at 1.0
Precision	bfloat16
Loss	Token-level cross-entropy, no label smoothing
Model selection	Lowest held-out loss (the final step; the loss was still falling)

No optimizer step was skipped for non-finite values. Gradients were clipped on 0.25% of steps.

Evaluation

Training results

Held-out loss kept falling to the last step, so it shows no sign of overfitting. Training loss steps down at the start of the second pass (step 46,495) because the model sees the examples again; the held-out loss does not step, which is the sign that matters. SARI on the 500 generated rewrites is noisy: it peaked at 79.0 at step 84,000 and ended at 78.4, a difference within the noise of a 500-example sample, so the final step (lowest held-out loss) was kept.

Held-out results

20,000 held-out examples for loss and accuracy; the first 500 of them for generation metrics. The held-out set comes from the same source as the training data, so it is more favorable than text from a new source. The "reference" column scores the held-out examples' own corrected targets with the same metrics. A blank cell means that figure was not measured for the reference rewrites.

Metric	No adapter	This adapter	Reference rewrites
Held-out loss ↓	0.304	0.0386
Perplexity ↓	1.355	1.039
Token accuracy	95.5%	98.7%
Stops correctly (EOS accuracy)	98.9%	99.7%
Calibration error (ECE) ↓	0.037	0.0009
SARI ↑	59.5	78.4	98.7
chrF vs the reference rewrite ↑	94.1	96.7
chrF vs the input	98.1	94.4	95.5
Output length ÷ input length	1.00	1.01	1.01
Output identical to the input	52.4%	1.4%	0%
Protected spans kept (numbers, URLs, emails, @mentions, code)	92.7%	93.1%	100%
Words not in the input	1.1%	2.9%
Repetition loops	0.6%	1.0%
Ends with a proper stop	100%	100%
Passes every quality check at once	93.8%	94.0%	96.6%

How to read this:

• "No adapter" is the base model with the adapter switched off. It does not correct text: more than half of its outputs are the input copied back. Because the inputs are already close to the corrected targets, its token accuracy (95.5%), its chrF scores and its "passes every check" score all look high. None of them show correction quality, so compare SARI and the loss. The held-out loss falls from 0.304 to 0.0386.
• SARI scores a rewrite by what it adds, keeps and removes compared with the reference. The adapter goes from 59.5 to 78.4. The reference rewrites score 98.7 because they are the reference itself, so that column is a ceiling, not a competitor.
• "Passes every quality check" requires, at once: non-empty, properly stopped, no loop, every protected span kept exactly, output at most 1.2x the input length, and chrF vs the input of at least 35. The base passes by copying, and the adapter ties it (94.0% against 93.8%), so this number does not separate them. It is below the 96.6% of the reference rewrites.
• Protected spans kept is the weak number: 6.9% of outputs changed at least one number, URL, email, mention or code span, against 0% for the reference rewrites. See Limitations.
• Words not in the input is low (2.9%), as it should be for a corrector.

Hand-written checks

To look at specific failure modes we ran the final adapter, with the default decoding, on small hand-written sets. None of these inputs appears in the training or held-out data (checked by exact match). The misspelling, preservation and repetition sets are in the GitHub repository under SFT_adapter_rephrase/eval_sets/professional/; the 26 extra sentences written for the examples table and the correct-text check are not published as a set. They are small checks, not a benchmark. We did not run other rewriting models on them.

Check	Result
Misspelled words (129 words, 31 sentences)	126 corrected (98%) by exact match; 128 if inflected forms count: "recieve" became "received" and "enviroment" became "environmental", both correct in context. The one miss is picnik.
Sentences with grammar errors (14)	11 fixed; 2 fixed only in part, leaving an error behind; 1 made worse (see Limitations)
Sentences that were already correct (12)	7 returned unchanged; 5 were edited (see Limitations)
Exact text that must survive (64 sentences, 138 items)	126 kept (91%) by exact string match; see below
Repetition and junk inputs (24)	1 runaway output (the digit 0 repeated); many others were turned into different text, see Limitations

Items that must survive, by kind:

Kept every time	Not always kept
names 13/13, product names 8/8, plain numbers 15/15, long numbers 6/6, phone numbers 4/4, acronyms 11/11, paths 2/2, @mentions 3/3, units 3/3, legal terms 2/2, lists 6/6, questions 2/2, multi-paragraph 2/2	URLs 7/8, dates 9/10, versions 4/6, code and config 10/11, IDs 5/6, emails 3/4, currency 2/3, security terms 6/8, negation 3/4, quoted speech 0/1

The negation and quoted-speech misses are not damage: the match is an exact string, and "NOT" was kept as NOT while the rest of the sentence changed around it, and a quotation had its first letter capitalized. The others are real changes to numbers, versions, URLs or addresses (next section).

Limitations

• It can make a sentence worse. Real outputs from the final adapter:

  Input	Output
  me and him goes to the market yesterday.	My and he went to the market yesterday.
  Between you and I, the new manager are not very good at his job.	Between you and I, the new manager is not very good at his job. (one error fixed, one left)
  The team have finished there part of the project and are waiting for you're feedback.	The team has finished their part of the project and are waiting for your feedback. (one fixed, one left)
  the weather was terible so we cancled the picnik	…so we canceled the picnik. (left misspelled)

  Heavily broken sentences are the risky ones. Read the result.

• Correct text still gets edited. The training data had almost no "nothing to fix" examples, and on 12 correct sentences it left 7 unchanged and edited 5. Real outputs:

  Input (already correct)	Output
  The meeting is scheduled for 3pm on Thursday.	The meeting is scheduled for 3 pm on Thursday.
  We shipped the feature last week, and the customers have responded well.	…and the customers responded well. (tense changed)
  I'll be on leave from 23 December to 2 January and back on the 3rd.	I'll be on leave from December 23 to January 2 and back on the 3rd.

  It also rewrites dates into month-first order in places. Do not run it as an automatic step on text you have not read.

• Numbers, dates, versions and code can change. Real outputs:

  Input	Output
  upgrade node from 18.17.1 to 20.11.0 and python 3.9 to 3.12	Upgrade node from 18.17.1 to 20.11.0 and Python 3.9 to 3:12.
  CVE-2024-3094 in xz-utils 5.6.0 and 5.6.1, patch to 5.4.6 now	CVE-2024-3094 in xz-utils 5.6.0 and 5:6.1, patch to 5,4.6 now.
  in config.yaml change replicas: 3 to replicas: 5	In config.yaml, change replicas: 3 to replicas. 5
  price changed from €1.299,00 to €1.149,00 for EU and £999 UK	Price changed from €1.299,00 to €1,149,00 for EU and £999 UK.
  send invoice to accounts_payable@zentech.example and cc j.smith@zentech.example asap	…cc j.smith@zen tech.example asap.
  hey i pushed the fix see github.com/sara-dev/echo/pull/77 pls review	Hey, I pushed the fix. See GitHub.com/sara-dev/echo/pull/77. Please review.
  ur booking ref is PNR 6E-QX7Z9K, flight 6E 2134 at gate B12	Your booking ref is PNR 6E-QX7Z9K, flight 6 E 2134 at gate B12.

  Version numbers and other dotted numbers are the weakest spot. Always compare dates, amounts, versions, identifiers and addresses against the original before you send or publish a correction.

• Junk input gets invented text. Of 24 repetitive or meaningless inputs, one (a run of 0 characters) hit the 384-token cap. Many others were turned into different text: eleven bye came back as bye bye bye, bye. Bye by, by, . . by,. ,., ,, ;,, and twelve ping as "Ping, ping, ping. … PINNING, PINSING, pings, pinging, and pinging." Do not feed it unfiltered or machine-generated text.

• Grammar, not style. It was trained to fix errors, not to simplify, shorten or formalize, and the outputs on our checks kept the input's wording. British spellings were left alone in them ("centre", "apologise").

• English only, and tuned for short to medium messages. The model accepts up to 2,048 tokens, but the code does not split long text, so split long documents yourself and correct them in parts.

• No factual checking. The adapter does not know whether the text is true.

Ethical considerations and risks

A corrector can change what a sentence says while making it look cleaner, and clean text looks more trustworthy than it may be. Do not use it to impersonate someone or to disguise the origin of content, and have a person review anything legal, medical, financial or safety-related. The license limits who may use the model and where (see below).

Benefits

A small download (73 MB on top of the base), local inference with no text sent to a server, and a result that you can compare with your original on the same screen.

License

KeelCore-750M-Grammar is released under the KeelCore License, Version 1.0 (Personal and Research Use). The license file controls; this section is a summary. The license covers the model and its adapters.

• Individuals may use KeelCore, at no cost, for their own writing (including writing for their own job or studies) and for noncommercial research, only through the official EchoMeBetter browser extension.
• Research publications that use KeelCore must cite it (see Citation).
• Commercial use, use by or for an organization, any use outside the official extension, and purchase of the model require a separate written agreement with the copyright holder (see Contact).
• The names "KeelCore" and "EchoMeBetter" are used by the copyright holder for this project; the license gives no right to use them.

Downloading the files from this repository does not grant any right beyond the license. Attributions for the base model are in the NOTICE file of KeelCore-750M.

Citation

@software{keelcore2026,
  author = {Saravanan A R},
  title  = {KeelCore: A 752M Encoder-Decoder Base Model for LoRA Adapters},
  year   = {2026},
  url    = {https://github.com/saravanan-a-r/KeelCore}
}

Contact

For commercial licensing, collaboration or anything else:

• Email: sararavi.ar@gmail.com
• GitHub: @saravanan-a-r
• LinkedIn: saravanan-a-r