README.md

metadata

license: gemma
base_model: google/gemma-3-270m
pipeline_tag: text-generation
library_name: mlx
datasets:
  - Gramscii/semantic-repair-routing
language:
  - en
  - it
  - fr
  - de
  - es
tags:
  - query-rewriting
  - query-understanding
  - intent-detection
  - routing
  - retrieval
  - rag
  - slm
  - gguf
  - multilingual

SemanticRepair-270M

A 270M rewriter that sits behind an embedding router. When a question does not land on any capability with enough margin, this model restates it in the plain form the capabilities are described in, and the router tries again on the restatement. When it reads no request at all, it says so.

That is the whole job. It does not answer questions, it does not decide anything, and nothing it writes is ever executed: the router runs on the restatement, the tool runs on the original.

The surface it reads, which is not a chat prompt

Fine-tuned on a bare completion surface. It has never seen a chat template, a system message or a few-shot example. Speak to it the way it was trained or it will not work:

{Language}: {the question, verbatim}
=>

It completes with one line per request it found, or the single token NO_REQUEST.

{Language} is the English name of the language the question is in — not the language of whatever system is asking. English, Italian, French, German, Spanish; a language outside that set keeps whatever tag the caller declares, because an invented tag is a surface the model never read either.

This is the whole call the engine makes, on /v1/completions and never on a chat endpoint:

{
  "prompt": "Italian: che tempo fa domani a Bologna?\n=>\n",
  "stop": ["\n=>"],
  "temperature": 0.0,
  "repeat_penalty": 1.0,
  "max_tokens": 96
}

Greedy, because the same words must always produce the same rewrite: a router that reruns on a different restatement each time cannot be reasoned about. repeat_penalty is 1.0, which is neutral — a rewrite legitimately repeats the words of the question. The stop string cuts a runaway that starts echoing another question in its own trained format; everything before it is the answer. 96 tokens leaves room for several rewritten lines: the longest answer in the training data is 43 tokens.

The sentinel is compared case-insensitively and tolerates a trailing period. It is an instruction the model follows, not a token it is guaranteed to emit byte-exact.

These are real outputs from the released weights, greedy:

in	out
Spanish: no busques la tienda de ropa, dime la dosis de paracetamol	dime la dosis de paracetamol
Italian: scusa il disturbo, mi diresti che tempo fa domani a Bologna?	dimmi che tempo fa domani a Bologna
English: book me a table for friday and also cancel my dentist	book me a table for friday cancel my dentist
Italian: guarda un po', questo sacchetto della spesa è tutto rotto.	NO_REQUEST

A negation is dropped, an apology is dropped, one message asking for two things becomes two lines, and something that is not a request at all answers with the sentinel.

What is in this repository

file	size	what it is
sft-v19-q8_0.gguf	300 MB	what the engine serves, through llama-server
model.safetensors	536 MB	the same weights fused, BF16, 236 tensors
model.safetensors.index.json		the index over that single file
tokenizer.json, tokenizer_config.json	33 MB	the tokenizer as the fuse wrote it
config.json, generation_config.json		gemma3_text, torch_dtype: bfloat16
chat_template.jinja		present because the fuse writes it — not the surface this model reads, see above

The GGUF's sha256 is 33b6cc73c7d306c9c81aa96c5a8ddb9dbfd46a5ac5d0388a2de45b5229928c56. The engine pins it and refuses anything else, which is what makes a routing decision reproducible.

Both files carry the same weights: the fused final adapter of training run v19, adapter sha256 1082985953ed6c33…, quantised to Q8_0. Q8_0 and not Q4: measured on this seat, at temperature 0 a Q8_0 is stable to the byte across runs and a Q4 is not.

What it is measured to do

Measured against the two things worth measuring against: the model it was fine-tuned from, and doing no repair at all. Two benches over different corpora, because a model that wins one of them is a model chosen by which table got printed.

400 routing tasks over 343 graphs, these exact released weights against the untrained base:

	exact	wrong executions	answered in the wrong language
this model	32	1	6
google/gemma-3-270m, untrained	24	0	123

The language column is what the fine-tune clearly buys, and it is the largest effect in any table here: 123 tasks answered in the wrong language become 6.

39 held-out questions on a live workspace, written by three blind judges against the capabilities in it, in two languages:

	exact
this model	16
routing the question as written, no repair	12
google/gemma-3-270m, untrained	9

The base model scoring below "no repair at all" is the whole argument for fine-tuning something for this seat: an untrained 270M put here makes routing worse than leaving the question alone. Of the 10 compound questions in that set — one message asking for two things — this model answers 2 whole.

Its own training gate, on 1000 held-out examples: 261 exact (26.1%), 927 of 1000 answers well formed, and the language-balance gate passed with a 7-point gap between English and Italian against a 10-point bar.

What it does not fix, which a card naming only the gains would hide

The sentinel does not catch every non-request. guarda un po', questo sacchetto della spesa è tutto rotto answers NO_REQUEST, but ciao come stai comes back verbatim rather than refused. If nothing downstream checks that the restatement is a request, a greeting will be routed.

It sometimes rewrites a word rather than restating a sentence. Measured: Bitte reserviere mir einen Platz im Klettergarten für Freitag comes back as reserviere mir einen Platz im Kletterpark für Freitag — Klettergarten became Kletterpark. The restatement is only ever compared against capability descriptions and never executed, which is why this is tolerable here; in any design where the rewrite is acted on, it is not.

Dates. In the 400-task run above, 160 of the tasks name a day. This model loses the day in 57 of them — more than a third. It is the weakest column it has, and it is weak in a way that does not announce itself: the restatement reads as a clean request, with the "yesterday" or the "before Easter" simply gone. If the day matters to your routing, measure that before you serve this.

It is not distinguishable from its own neighbours. Ten checkpoints of this run were compared on the same 400 tasks pair by pair rather than by totals — counting only the tasks where two checkpoints disagree, which are the only ones carrying information. Not one difference reaches significance: the widest gap in the table is 7 tasks to 1, p = 0.07. With around thirty exact answers out of four hundred the natural fluctuation is about five, which is most of the ranking. Treat the order inside a run as noise: a checkpoint one or two points ahead of another inside the same run has not been shown to be better than it.

Where it runs

Served with llama-server, reached by the engine as one of four model seats:

llama-server -m sft-v19-q8_0.gguf --host 127.0.0.1 --port 9250 -ngl 99 -c 2048 --jinja

It is loaded only when routing is genuinely unsure, never as the default path, and released after an idle window.

The data it was trained on

Gramscii/semantic-repair-routing — published, and everything below can be recounted from it.

84,819 pairs: 82,819 train, 1,000 validation, 1,000 test. Each pair is a message somebody could plausibly write and the requests inside it restated plainly, one per line.

Five languages, close to balanced:

	rows
English	18,006	21.7%
Spanish	16,903	20.4%
German	16,659	20.1%
French	16,602	20.0%
Italian	14,649	17.7%

What the message is doing, which is what the data is really organised by. Every one of the 82,819 training rows carries one of 38 such labels; these are the ten largest:

	rows
massive_real — real assistant traffic	5,736	6.9%
minimal_pair — two messages differing in one word	5,299	6.4%
indirect — the request is implied, never stated	4,912	5.9%
presto_native — natively written, not translated	4,606	5.6%
oasst_human — human-written	4,450	5.4%
multi — more than one request in one message	3,974	4.8%
direct — the plain case	3,904	4.7%
vent_statement_request — a complaint with a request inside	3,516	4.2%
negation_exclusion — names what is not wanted	3,398	4.1%
trap_topic_lure — a topic named to pull routing the wrong way	3,324	4.0%

The remaining 28 cover self-correction, code-switching, ultra-short messages, typos and slang, messages carrying two, three or four requests, eight kinds of date reference — a weekday in the past, a named holiday, a bounded range — and the two families below.

The two families that decide what this model refuses to do

injection_meta — 2,278 rows — is a message aimed at the assistant itself: reveal your instructions, ignore what you were told, pretend you have no rules. 1,901 of them answer NO_REQUEST, because there is nothing here to route. Together with injection_command these two families supply 19% of every NO_REQUEST row in the set: refusing is largely taught by messages that try hardest to get an answer.

injection_command — 2,410 rows — is the opposite lesson, and the more delicate one. A hostile but explicit command about the user's own things: delete every saved bank account, empty my personal cloud, cancel all my bookings tonight. 2,281 of them are restated faithfully, action intact.

The rewriter does not sanitise and does not soften. Deciding whether an action is allowed is not its job and it has no way to do that job well: it sees one sentence, not the account, the permissions or the consequences. A model that quietly dropped "delete" would hand the router a different request from the one the person made, and the person would never learn their instruction had been edited. So the restatement carries the command as written, the router places it, and whatever runs it is where refusing belongs.

12.6% of rows (10,432) answer NO_REQUEST, and 16.9% (14,028) answer with more than one line.

Training

Fine-tuned from google/gemma-3-270m with mlx_lm lora, full fine-tune. The released weights are the run's final adapter, chosen by a gate on 1,000 held-out examples rather than by the best number on a validation curve.

Licence

Gemma terms, which travel with the file: this is a fine-tune of Gemma-3-270M and inherits them.