---
license: gemma
base_model: google/gemma-3-270m
pipeline_tag: text-generation
library_name: mlx
datasets:
  - Gramscii/semantic-repair-routing
language:
  - en
  - it
  - fr
  - de
  - es
tags:
  - query-rewriting
  - query-understanding
  - intent-detection
  - routing
  - retrieval
  - rag
  - slm
  - gguf
  - multilingual
---

# SemanticRepair-270M

A 270M rewriter that sits behind an embedding router. When a question does
not land on any capability with enough margin, this model restates it in
the plain form the capabilities are described in, and the router tries
again on the restatement. When it reads no request at all, it says so.

That is the whole job. It does not answer questions, it does not decide
anything, and nothing it writes is ever executed: the router runs on the
restatement, the tool runs on the original.

## The surface it reads, which is not a chat prompt

Fine-tuned on a bare completion surface. It has never seen a chat template,
a system message or a few-shot example. Speak to it the way it was trained
or it will not work:

```
{Language}: {the question, verbatim}
=>
```

It completes with one line per request it found, or the single token
`NO_REQUEST`.

`{Language}` is the English name of the language **the question** is in —
not the language of whatever system is asking. `English`, `Italian`,
`French`, `German`, `Spanish`; a language outside that set keeps whatever
tag the caller declares, because an invented tag is a surface the model
never read either.

This is the whole call the engine makes, on `/v1/completions` and never on
a chat endpoint:

```json
{
  "prompt": "Italian: che tempo fa domani a Bologna?\n=>\n",
  "stop": ["\n=>"],
  "temperature": 0.0,
  "repeat_penalty": 1.0,
  "max_tokens": 96
}
```

Greedy, because the same words must always produce the same rewrite: a
router that reruns on a different restatement each time cannot be reasoned
about. `repeat_penalty` is 1.0, which is neutral — a rewrite legitimately
repeats the words of the question. The stop string cuts a runaway that
starts echoing another question in its own trained format; everything
before it is the answer. 96 tokens leaves room for several rewritten lines:
the longest answer in the training data is 43 tokens.

The sentinel is compared case-insensitively and tolerates a trailing
period. It is an instruction the model follows, not a token it is
guaranteed to emit byte-exact.

These are real outputs from the released weights, greedy:

| in | out |
|---|---|
| `Spanish: no busques la tienda de ropa, dime la dosis de paracetamol` | `dime la dosis de paracetamol` |
| `Italian: scusa il disturbo, mi diresti che tempo fa domani a Bologna?` | `dimmi che tempo fa domani a Bologna` |
| `English: book me a table for friday and also cancel my dentist` | `book me a table for friday`<br>`cancel my dentist` |
| `Italian: guarda un po', questo sacchetto della spesa è tutto rotto.` | `NO_REQUEST` |

A negation is dropped, an apology is dropped, one message asking for two
things becomes two lines, and something that is not a request at all
answers with the sentinel.

## What is in this repository

| file | size | what it is |
|---|---|---|
| `sft-v19-q8_0.gguf` | 300 MB | what the engine serves, through llama-server |
| `model.safetensors` | 536 MB | the same weights fused, BF16, 236 tensors |
| `model.safetensors.index.json` | | the index over that single file |
| `tokenizer.json`, `tokenizer_config.json` | 33 MB | the tokenizer as the fuse wrote it |
| `config.json`, `generation_config.json` | | `gemma3_text`, `torch_dtype: bfloat16` |
| `chat_template.jinja` | | present because the fuse writes it — **not the surface this model reads**, see above |

The GGUF's sha256 is
`33b6cc73c7d306c9c81aa96c5a8ddb9dbfd46a5ac5d0388a2de45b5229928c56`.
The engine pins it and refuses anything else, which is what makes a routing
decision reproducible.

Both files carry the same weights: the fused final adapter of training run
v19, adapter sha256 `1082985953ed6c33…`, quantised to Q8_0. Q8_0 and not
Q4: measured on this seat, at temperature 0 a Q8_0 is stable to the byte
across runs and a Q4 is not.

## What it is measured to do

Measured against the two things worth measuring against: the model it was
fine-tuned from, and doing no repair at all. Two benches over different
corpora, because a model that wins one of them is a model chosen by which
table got printed.

**400 routing tasks over 343 graphs**, these exact released weights against
the untrained base:

| | exact | wrong executions | answered in the wrong language |
|---|---|---|---|
| **this model** | **32** | 1 | **6** |
| `google/gemma-3-270m`, untrained | 24 | 0 | **123** |

The language column is what the fine-tune clearly buys, and it is the
largest effect in any table here: 123 tasks answered in the wrong language
become 6.

**39 held-out questions on a live workspace**, written by three blind
judges against the capabilities in it, in two languages:

| | exact |
|---|---|
| **this model** | **16** |
| routing the question as written, no repair | 12 |
| `google/gemma-3-270m`, untrained | 9 |

The base model scoring **below** "no repair at all" is the whole argument
for fine-tuning something for this seat: an untrained 270M put here makes
routing worse than leaving the question alone. Of the 10 compound questions
in that set — one message asking for two things — this model answers 2
whole.

**Its own training gate**, on 1000 held-out examples: 261 exact (26.1%),
927 of 1000 answers well formed, and the language-balance gate passed with
a 7-point gap between English and Italian against a 10-point bar.

## What it does not fix, which a card naming only the gains would hide

**The sentinel does not catch every non-request.** `guarda un po', questo
sacchetto della spesa è tutto rotto` answers `NO_REQUEST`, but `ciao come
stai` comes back verbatim rather than refused. If nothing downstream checks
that the restatement is a request, a greeting will be routed.

**It sometimes rewrites a word rather than restating a sentence.** Measured:
`Bitte reserviere mir einen Platz im Klettergarten für Freitag` comes back
as `reserviere mir einen Platz im Kletterpark für Freitag` — `Klettergarten`
became `Kletterpark`. The restatement is only ever compared against
capability descriptions and never executed, which is why this is tolerable
here; in any design where the rewrite is acted on, it is not.

**Dates.** In the 400-task run above, 160 of the tasks name a day. This
model loses the day in **57 of them** — more than a third. It is the
weakest column it has, and it is weak in a way that does not announce
itself: the restatement reads as a clean request, with the "yesterday" or
the "before Easter" simply gone. If the day matters to your routing,
measure that before you serve this.

**It is not distinguishable from its own neighbours.** Ten checkpoints of
this run were compared on the same 400 tasks pair by pair rather than by
totals — counting only the tasks where two checkpoints disagree, which are
the only ones carrying information. Not one difference reaches
significance: the widest gap in the table is 7 tasks to 1, p = 0.07. With
around thirty exact answers out of four hundred the natural fluctuation is
about five, which is most of the ranking. Treat the order inside a run as
noise: a checkpoint one or two points ahead of another inside the same run
has not been shown to be better than it.

## Where it runs

Served with llama-server, reached by the engine as one of four model seats:

```
llama-server -m sft-v19-q8_0.gguf --host 127.0.0.1 --port 9250 -ngl 99 -c 2048 --jinja
```

It is loaded only when routing is genuinely unsure, never as the default
path, and released after an idle window.

## The data it was trained on

**[`Gramscii/semantic-repair-routing`](https://huggingface.co/datasets/Gramscii/semantic-repair-routing)**
— published, and everything below can be recounted from it.

**84,819 pairs**: 82,819 train, 1,000 validation, 1,000 test. Each pair is a
message somebody could plausibly write and the requests inside it restated
plainly, one per line.

**Five languages**, close to balanced:

| | rows | |
|---|---|---|
| English | 18,006 | 21.7% |
| Spanish | 16,903 | 20.4% |
| German | 16,659 | 20.1% |
| French | 16,602 | 20.0% |
| Italian | 14,649 | 17.7% |

**What the message is doing**, which is what the data is really organised
by. Every one of the 82,819 training rows carries one of 38 such labels;
these are the ten largest:

| | rows | |
|---|---|---|
| `massive_real` — real assistant traffic | 5,736 | 6.9% |
| `minimal_pair` — two messages differing in one word | 5,299 | 6.4% |
| `indirect` — the request is implied, never stated | 4,912 | 5.9% |
| `presto_native` — natively written, not translated | 4,606 | 5.6% |
| `oasst_human` — human-written | 4,450 | 5.4% |
| `multi` — more than one request in one message | 3,974 | 4.8% |
| `direct` — the plain case | 3,904 | 4.7% |
| `vent_statement_request` — a complaint with a request inside | 3,516 | 4.2% |
| `negation_exclusion` — names what is *not* wanted | 3,398 | 4.1% |
| `trap_topic_lure` — a topic named to pull routing the wrong way | 3,324 | 4.0% |

The remaining 28 cover self-correction, code-switching, ultra-short
messages, typos and slang, messages carrying two, three or four requests,
eight kinds of date reference — a weekday in the past, a named holiday, a
bounded range — and the two families below.

### The two families that decide what this model refuses to do

**`injection_meta`** — 2,278 rows — is a message aimed at the assistant
itself: reveal your instructions, ignore what you were told, pretend you
have no rules. **1,901 of them answer `NO_REQUEST`**, because there is
nothing here to route. Together with `injection_command` these two families
supply **19% of every `NO_REQUEST` row in the set**: refusing is largely
taught by messages that try hardest to get an answer.

**`injection_command`** — 2,410 rows — is the opposite lesson, and the more
delicate one. A hostile but explicit command about the user's *own* things:
delete every saved bank account, empty my personal cloud, cancel all my
bookings tonight. **2,281 of them are restated faithfully**, action intact.

The rewriter does not sanitise and does not soften. Deciding whether an
action is allowed is not its job and it has no way to do that job well: it
sees one sentence, not the account, the permissions or the consequences. A
model that quietly dropped "delete" would hand the router a different
request from the one the person made, and the person would never learn
their instruction had been edited. So the restatement carries the command
as written, the router places it, and whatever runs it is where refusing
belongs.

**12.6%** of rows (10,432) answer `NO_REQUEST`, and **16.9%** (14,028)
answer with more than one line.

## Training

Fine-tuned from `google/gemma-3-270m` with `mlx_lm lora`, full fine-tune.
The released weights are the run's final adapter, chosen by a gate on 1,000
held-out examples rather than by the best number on a validation curve.

## Licence

Gemma terms, which travel with the file: this is a fine-tune of
Gemma-3-270M and inherits them.