Blame for README.md

---
license: other
license_name: keelcore-license-1.0
license_link: LICENSE
language:
- en
base_model: KeelCore/KeelCore-750M
base_model_relation: adapter
pipeline_tag: text-generation
tags:
- lora
- adapter
- text-rewriting
- concise-writing
- text-shortening
- encoder-decoder
- onnx
---

<p align="center">
  <img src="banner.png" alt="KeelCore Concise: wordy text in, tight writing out" width="100%">
</p>

<p align="center">
  <b><a href="https://huggingface.co/KeelCore/KeelCore-750M" target="_blank" rel="noopener noreferrer">Base model</a></b> &nbsp;|&nbsp;
  <b><a href="https://saravanan-a-r.github.io/KeelCore/" target="_blank" rel="noopener noreferrer">Technical report</a></b> &nbsp;|&nbsp;
  <b><a href="https://github.com/saravanan-a-r/KeelCore" target="_blank" rel="noopener noreferrer">GitHub</a></b> &nbsp;|&nbsp;
  <b><a href="https://huggingface.co/KeelCore/KeelCore-750M-Concise/blob/main/LICENSE" target="_blank" rel="noopener noreferrer">License</a></b> &nbsp;|&nbsp;
  <b><a href="#contact">Contact</a></b>
</p>

---

# KeelCore-750M-Concise

**KeelCore-750M-Concise** is a LoRA adapter for <a href="https://huggingface.co/KeelCore/KeelCore-750M" target="_blank" rel="noopener noreferrer">KeelCore-750M</a> that rewrites wordy text into shorter, tighter writing. It cuts filler phrases and padding, keeps the facts, and keeps names, links and numbers where it can. It is a 73 MB file, not a full model: it needs the frozen base model to run.

* **Task:** tightening English text (removing filler and padding from emails, messages, notes and short documents)
* **Size:** rank-16 LoRA, 18.29M trainable parameters (2.43% of the base)
* **Runs:** on CPU or GPU with PyTorch, or in a browser through ONNX Runtime Web
* **Terms of use:** <a href="https://huggingface.co/KeelCore/KeelCore-750M-Concise/blob/main/LICENSE" target="_blank" rel="noopener noreferrer">KeelCore License, Version 1.0</a> (see [License](#license))
* **Author:** <a href="https://www.linkedin.com/in/saravanan-a-r/" target="_blank" rel="noopener noreferrer">Saravanan A R</a>

## Model Information

### Description

You give the adapter a piece of text and it returns the same message in fewer words. On the held-out set, outputs are on average 20% shorter than the input, and 97% of them are shorter. The base model is an encoder–decoder, so it reads the whole input first and can copy names, URLs and other exact text straight into the output. The adapter teaches it the rewriting task. It is not a chat model, it does not answer questions or follow instructions in the text, and it is not a summarizer: it keeps each point and drops the words around it. It only rewrites the text it is given.

### Examples

These come from the final adapter, run with the default settings described under [Usage](#usage). The inputs are not from the training data (see [Hand-written checks](#hand-written-checks)).

| Input | Output |
|---|---|
| Due to the fact that it was raining, we decided to stay at home. | It was raining, so we stayed at home. |
| In the event that the server goes down, please contact the support team as soon as possible. | If the server goes down, contact support immediately. |
| Please be advised that the office will be closed on Monday for the purpose of maintenance. | Please note the office will close on Monday for maintenance. |
| It is important to note that the deadline has been extended until the end of next week. | The deadline has been extended until the end of next week. |
| The report that was written by the finance team contains a lot of useful information about the budget. | The finance team's report contains useful information about the budget. |
| Visit https://example.com/docs/start in order to read the setup guide that we have prepared for you. | Visit https://example.com/docs/start to read the setup guide we prepared for you. |
| Call 1-800-555-0199 or send an email to support@example.com if you are still experiencing any kind of problem with the device. | Call 1-800-555-0199 or email support@example.com if you still have a device issue. |

### Inputs and outputs

| Direction | Description |
|---|---|
| **Input** | Plain English text. The code adds the frame `<style:concise> <text_to_rewrite> … </text_to_rewrite> </s>` for you. |
| **Output** | The rewritten text, up to 384 new tokens with the default settings. |

### Intended use

* Trimming drafts: emails, status updates, tickets, notes and chat messages that say too much.
* A rewriting step inside an application, on device or in a browser, where the text should not leave the user's machine.
* A starting point for researchers who study small adapters on a shared base model.

**Out of scope.** Text where every word or character must survive (legal clauses, code, configuration, financial figures, quotations) unless a person checks the result. The style removes words, so it can remove a detail you needed. Short text that is already tight, which it mostly returns unchanged. Languages other than English. Generating text from scratch, or summarizing a long document into a few points.

## Usage

The adapter file is in a KeelCore format (`adapter.safetensors` with `adapter_config.json`), not the PEFT format, and it is loaded by the code in the <a href="https://github.com/saravanan-a-r/KeelCore" target="_blank" rel="noopener noreferrer">KeelCore GitHub repository</a>. It checks a fingerprint of the base weights and refuses to load onto any base other than KeelCore-750M.

```bash
git clone https://github.com/saravanan-a-r/KeelCore
pip install torch safetensors sentencepiece pyyaml huggingface_hub
```

```python
import sys
from huggingface_hub import snapshot_download

sys.path.insert(0, "KeelCore/SFT_adapter_rephrase")
from src.inference import RephraseAdapter

# The base needs only these four files; the ONNX folder is not required.
base = snapshot_download(
    "KeelCore/KeelCore-750M",
    allow_patterns=["config.json", "model.safetensors", "spm.model", "token_map.json"],
)
adapter = snapshot_download("KeelCore/KeelCore-750M-Concise", ignore_patterns=["onnx/*"])  # PyTorch files only

rephraser = RephraseAdapter.load(adapter, base_checkpoint=base, tokenizer_dir=base, device="cpu")  # or "cuda"
print(rephraser.rephrase(["I just wanted to reach out and let you know that the meeting has been moved to 3pm."]))
```

`rephrase()` takes a list of texts and returns a list of rewrites. The base loads in float32 (about 3 GB of memory).

**Decoding.** The defaults are greedy decoding, a ban on repeating any 3-token sequence, and at most 384 new tokens. The tokenizer writes every digit as its own token, so the ban would force numbers to change (`50000` becomes `5000`). The default therefore exempts the ten digit tokens from the ban. If you write your own decoding loop, keep that exemption.

**Download only what you need.** Both formats are in this repository, in separate places, so you never have to fetch both. For PyTorch, use `ignore_patterns=["onnx/*"]` as above (73 MB). For the browser or ONNX Runtime, use `snapshot_download("KeelCore/KeelCore-750M-Concise", allow_patterns=["onnx/*"])` (73 MB), or request the two `.onnx_adapter` files directly by URL. The base model follows the same rule: its ONNX export lives in `onnx/int8/` and is not downloaded by the PyTorch snippet.

**In the browser.** `onnx/` holds this adapter converted to ONNX Runtime adapter files. They load on the int8 ONNX export in the base repository (`onnx/int8/`) through ONNX Runtime's `LoraAdapter` support (`activeLoraAdapters` run option). The base graphs, tokenizer and `model.json` are in `onnx/int8/` of the base repository; `onnx/adapter.json` here lists the two adapter files with their hashes and the base they fit. The EchoMeBetter extension in the GitHub repository is a working example.

### Repository contents

| File | Size | Description |
|---|---:|---|
| `adapter.safetensors` | 73 MB | LoRA weights (fp32) and the learned embeddings of the three frame tokens |
| `adapter_config.json` | 2 KB | LoRA settings, frame tokens and the fingerprint of the base it needs |
| `config.json` | 1 KB | Names the base model and the style. The Hub counts a download of this repository when this file is requested |
| `onnx/encoder.onnx_adapter`, `onnx/decoder.onnx_adapter` | 37 MB each | The same adapter for the ONNX export of the base (encoder and decoder graphs) |
| `onnx/adapter.json` | 1 KB | The adapter's manifest for ONNX runners: style token, the two files with sizes and SHA-256 hashes, and the base it needs |
| `training_curves.png`, `banner.png` | | Images used on this page |
| `LICENSE` | | KeelCore License, Version 1.0 |

### Adapter details

| | |
|---|---|
| Base model | <a href="https://huggingface.co/KeelCore/KeelCore-750M" target="_blank" rel="noopener noreferrer">KeelCore-750M</a>, pretraining step 90,000 (weights unchanged) |
| Method | LoRA with rank-stabilized scaling, rank 16, alpha 8 (scale 2.0), dropout 0.05 |
| Where | Every linear layer of the encoder and decoder: self-attention, cross-attention (`q`, `k`, `v`, `o`) and feed-forward (`gate`, `up`, `down`), 432 LoRA pairs |
| Also trained | Embeddings of the three frame tokens `<style:concise>`, `<text_to_rewrite>`, `</text_to_rewrite>` |
| Trainable parameters | 18,287,616 |

## Phase 2 update (November 2026)

This version replaces the first release. The first adapter changed dates and other exact details too often
(`23rd dec` was rewritten as `December 23rd`, and sometimes as the wrong date, `December 2nd`). Phase 2 continued training the same
adapter (same size, same base) for 1,389 more steps on 20,768 new examples: realistic casual messages
full of dates, times, links and numbers, rewritten by a larger model and kept only if every date, number, link,
email, @handle, ID and code span survived exactly and a second model check passed. They were mixed one to one with
clean examples from the first training set, so the adapter keeps what it learned before. A first phase-2 attempt
taught two new habits (writing `50000` as `50,000`, and capitalising emails such as `j.smith@…` like a name); the
training data was repaired and the adapter retrained before this release.

Measured with the same decoding as the browser extension, phase-1 adapter against this one, on the same inputs. None
of the inputs was trained on. The sets are in the GitHub repository under `SFT_adapter_rephrase/eval_sets/`.

| | Phase 1 (before) | **Phase 2 (now)** |
|---|---:|---:|
| Dates changed in value, 96 casual messages with dates ↓ | 1 | **0** |
| Dates kept but written differently (`23rd dec` → `December 23rd`) ↓ | 17 | **0** |
| 500 new held-out messages: a date changed in value ↓ | 49 | **43** |
| same: a date written differently ↓ | 79 | **21** |
| same: another fact lost (number, link, email, ID) ↓ | 66 | **55** |
| same: spelling/shorthand left or other clean-up miss ↓ | 54 | **15** |
| 90 messages with amounts, IDs, versions, emails, phones: a fact lost ↓ | 12 | **1** |
| 60 messages with plain numbers that must keep their digits (ports, PINs, `50000`): changed ↓ | 0 | **0** |
| 45 messages with emails, @handles, paths, commands: not kept exactly ↓ | 4 | **3** |
| Exact-text stress set (64 sentences): items lost ↓ | 8 | **8** |
| Misspelled words corrected (129) | 120/129 | **123/129** |
| Original held-out set (20,000 rows): loss ↓ | 0.402 | **0.407** |
| same, 500 generated: protected spans kept ↑ | 94.8% | **96.0%** |
| same: passes every quality check ↑ | 93.8% | **93.6%** |
| same: SARI ↑ | 66.3 | **65.2** |

The same checks on the int8 ONNX model that the browser extension runs give the same picture. The loss on the
original held-out set rose slightly (1-3%): the wording moved slightly toward the new training examples, while the
generated rewrites on that set kept as many facts or more. Version numbers, code and some email domains can still be
changed (see [Limitations](#limitations)); most of those come from the decoding rule that blocks repeated word
pieces, not from the training.

## Training

The adapter was trained on about 2.03 million wordy-to-concise examples for 2 passes (126,846 optimizer steps), with about 20,000 further examples held out for evaluation. The base weights stayed frozen. Every training target is different from its input; none is a copy.

| Setting | Value |
|---|---|
| Effective batch size | 32 examples |
| Learning rate | 3e-4, 100 warm-up steps, cosine decay to 3e-5 |
| Optimizer | AdamW, no weight decay, gradient clipping at 1.0 |
| Precision | bfloat16 |
| Loss | Token-level cross-entropy, no label smoothing |
| Model selection | Lowest held-out loss (the final step; the loss was still falling) |

No optimizer step was skipped for non-finite values. Gradients were clipped on 2.6% of steps.

## Evaluation

### Training results

*Phase 1 (the first 126,846-step run). Phase 2 added 1,389 steps on top; see [Phase 2 update](#phase-2-update-november-2026).*

<p align="center">
  <img src="training_curves.png" alt="Training loss, held-out loss and SARI over 126,846 steps" width="100%">
</p>

Held-out loss kept falling to the last step, so it shows no sign of overfitting. Training loss steps down at the start of the second pass (step 63,423) because the model sees the examples again; the held-out loss does not step, which is the sign that matters. SARI on the 500 generated rewrites is noisy: over the last 12,700 steps it moves between 65.1 and 66.1, and the final value is also the best one.

### Held-out results

*Measured on the phase-1 adapter, before the phase-2 update; see [Phase 2 update](#phase-2-update-november-2026) for the current adapter against it.*

19,999 held-out examples for loss and accuracy; the first 500 of them for generation metrics. The held-out set comes from the same source as the training data, so it is more favorable than text from a new source. The "reference" column scores the held-out examples' own target rewrites with the same metrics. A blank cell means that figure was not measured for the reference rewrites.

| Metric | No adapter | **This adapter** | Reference rewrites |
|---|---:|---:|---:|
| Held-out loss ↓ | 2.528 | **0.399** | |
| Perplexity ↓ | 12.52 | **1.490** | |
| Token accuracy | 71.6% | **87.7%** | |
| Stops correctly (EOS accuracy) | 84.0% | **99.3%** | |
| Calibration error (ECE) ↓ | 0.228 | **0.008** | |
| SARI ↑ | 47.4 | **66.1** | 98.4 |
| chrF vs the reference rewrite ↑ | 69.0 | **76.1** | |
| chrF vs the input | 98.2 | 65.8 | 60.6 |
| Output length ÷ input length | 1.00 | 0.80 | 0.77 |
| Output identical to the input | 57.2% | **0.4%** | 0% |
| Output shorter than the input | 16.2% | **97.0%** | |
| Protected spans kept (numbers, URLs, emails, @mentions, `code`) | 94.7% | 94.7% | 98.2% |
| Words not in the input | 1.0% | 11.9% | |
| Repetition loops | 1.4% | **0.0%** | |
| Ends with a proper stop | 98.6% | 99.8% | |
| Passes every quality check at once | 77.8% | **93.2%** | 95.8% |

How to read this:

* **"No adapter" is the base model with the adapter switched off.** It does not shorten text: more than half of its outputs are the input copied back, and only 16% are shorter. That is why its "meaning kept" score (chrF vs the input) is high. It does not show rewriting quality, so compare SARI and the loss.
* **SARI** scores a rewrite by what it adds, keeps and removes compared with the reference. The adapter goes from 47.4 to 66.1. The reference rewrites score 98.4 because they are the reference itself, so that column is a ceiling, not a competitor.
* **"Passes every quality check"** requires, at once: non-empty, properly stopped, no loop, every protected span kept exactly, output no longer than the input, and chrF vs the input of at least 35. The adapter passes 93.2% against 95.8% for the reference rewrites.
* **Protected spans kept** is the same as for the base model (94.7%), which mostly copies. 5.3% of outputs changed at least one number, URL, email, mention or code span. See [Limitations](#limitations).
* **Words not in the input** is 11.9%: it prefers a shorter word ("immediately" for "as soon as possible"), so some new words are expected.

### Hand-written checks

*Measured on the phase-1 adapter, before the phase-2 update.*

To look at specific failure modes we ran the final adapter, with the default decoding, on small hand-written sets. None of these inputs appears in the training or held-out data (checked by exact match; one extra sentence that matched a training target was replaced). The wordy-sentence set (40), and the preservation and repetition sets that were written for all the adapters, are in the GitHub repository under `SFT_adapter_rephrase/eval_sets/`; the 14 extra sentences written for the examples table are not published as a set. They are small checks, not a benchmark. We did not run other rewriting models on them.

| Check | Result |
|---|---|
| Wordy sentences, easy to hard (40 + 14 extra) | Outputs are on average 30% shorter than the input; 47 of the 54 are shorter, 4 are returned unchanged |
| Exact text that must survive (64 sentences, 138 items) | 125 kept (91%) by exact string match; see below |
| Repetition and junk inputs (24) | 1 runaway output (the digit `0` repeated, which hit the 384-token cap); see [Limitations](#limitations) |

Items that must survive, by kind:

| Kept every time | Not always kept |
|---|---|
| URLs 8/8, names 13/13, product names 8/8, plain numbers 15/15, long numbers 6/6, phone numbers 4/4, acronyms 11/11, paths 2/2, @mentions 3/3, units 3/3, legal terms 2/2, lists 6/6, questions 2/2, multi-paragraph 2/2 | dates 8/10, versions 4/6, code and config 10/11, IDs 5/6, emails 3/4, currency 2/3, security terms 6/8, negation 2/4, quoted speech 0/1 |

The two negation misses and the quoted-speech miss are wording changes, not damage: the match is an exact string, and "did not approve" became "didn't approve" while a quotation had its first letter capitalized. The other misses are reformatted dates and times or damaged numbers, versions and addresses (next section).

## Limitations

* **It removes information on purpose, and sometimes the wrong information.** Real outputs from the final adapter:

  | Input | Output |
  |---|---|
  | Our vendor has decided to raise their prices on us, and for that reason we are now in the process of looking for a cheaper alternative that costs less. | Our vendor raised their prices, so we're now looking for a **cheaper, cheaper** alternative. |
  | Because a fire drill took place during the exam, the students who were taking it were given a bit of extra time in order to finish. | **During the fire drill**, students were given extra time to finish the exam. (the cause is now stated as a time) |

  Compare every rewrite with the original before you use it.
* **Numbers, versions and code can change.** Dates are kept far more reliably since the phase-2 update. Real outputs from the current adapter:

  | Input | Output |
  |---|---|
  | upgrade node from 18.17.1 to 20.11.0 and python 3.9 to 3.12 | Upgrade node from 18.17.1 to 20.11.0 and Python 3.9 to **312** |
  | bug only in build 5.6.1-rc2, not in 5.6.0 | Bug only in build 5.6.1-rc2, not **56.0** |
  | price changed from €1.299,00 to €1.149,00 for EU and £999 UK | Price changed from €1.299,00 to **€11.149,00** EU and £999 UK |
  | CVE-2024-3094 in xz-utils 5.6.0 and 5.6.1, patch to 5.4.6 now | CVE-2024-3094 in xz-utils 5.6.0 and **56.1**; patch to **54.6** now. |
  | in config.yaml change replicas: 3 to replicas: 5 | In config.yaml change replicas: 3 to **replicas.5**. |
  | send invoice to accounts_payable@zentech.example and cc j.smith@zentech.example asap | …cc j.smith@**zen Tech**.example ASAP. |
  | ur booking ref is PNR 6E-QX7Z9K, flight 6E 2134 at gate B12 | Your booking reference is PNR 6E-QX7Z9K, flight **6 E** 2134 at gate B12. |

  Version numbers and other dotted numbers are the weakest spot: the dots are sometimes dropped or moved. Always compare dates, amounts, versions, identifiers and addresses against the original before you send or publish a rewrite.
* **Runaway text on junk input.** Of 24 repetitive or meaningless inputs, 23 came back short or unchanged, as intended ("no" repeated eleven times became "No"). One failed: a run of `0` characters came back as a longer run that hit the 384-token cap. Do not feed it unfiltered or machine-generated text.
* **Short and already tight text is mostly left alone**, and a few short inputs come back unchanged or only re-capitalized. This is correct behavior, not a failure.
* **Tone can shift.** Cutting hedges and politeness phrases ("Please be advised", "I would like to take this opportunity") makes text blunter. Check that the result still sounds the way you want.
* **English only**, and tuned for short to medium messages. The model accepts up to 2,048 tokens, but the code does not split long text, so split long documents yourself and rewrite them in parts.
* **No factual checking.** The adapter does not know whether the text is true.

### Ethical considerations and risks

A rewriter that cuts words can cut a caveat, a condition or a negation, and the shorter text then reads as more certain than the original. Do not use it to disguise the origin of content, and have a person review anything legal, medical, financial or safety-related. The license limits who may use the model and where (see below).

### Benefits

A small download (73 MB on top of the base), local inference with no text sent to a server, and a result that you can compare with your original on the same screen.

## License

KeelCore-750M-Concise is released under the **<a href="https://huggingface.co/KeelCore/KeelCore-750M-Concise/blob/main/LICENSE" target="_blank" rel="noopener noreferrer">KeelCore License, Version 1.0</a>** (Personal and Research Use). The license file controls; this section is a summary. The license covers the model and its adapters.

* **Individuals** may use KeelCore, at no cost, for their own writing (including writing for their own job or studies) and for noncommercial research, only through the official EchoMeBetter browser extension.
* **Research publications** that use KeelCore must cite it (see [Citation](#citation)).
* **Commercial use**, use by or for an organization, any use outside the official extension, and purchase of the model require a separate written agreement with the copyright holder (see [Contact](#contact)).
* The names "KeelCore" and "EchoMeBetter" are used by the copyright holder for this project; the license gives no right to use them.

Downloading the files from this repository does not grant any right beyond the license. Attributions for the base model are in the `NOTICE` file of <a href="https://huggingface.co/KeelCore/KeelCore-750M" target="_blank" rel="noopener noreferrer">KeelCore-750M</a>.

## Citation

```bibtex
@software{keelcore2026,
  author = {Saravanan A R},
  title  = {KeelCore: A 752M Encoder-Decoder Base Model for LoRA Adapters},
  year   = {2026},
  url    = {https://github.com/saravanan-a-r/KeelCore}
}
```

## Contact

For commercial licensing, collaboration or anything else:

* **Email:** sararavi.ar@gmail.com
* **GitHub:** <a href="https://github.com/saravanan-a-r" target="_blank" rel="noopener noreferrer">@saravanan-a-r</a>
* **LinkedIn:** <a href="https://www.linkedin.com/in/saravanan-a-r/" target="_blank" rel="noopener noreferrer">saravanan-a-r</a>