---
license: gemma
base_model: google/gemma-3-4b-it
language:
- en
tags:
- mlx
- gemma3
- text-simplification
- cefr
- lora
- A2
- gspo
- rl
pipeline_tag: text-generation
---

# Claro 4B

*Claro* is a fine-tuned **Gemma 3 4B Instruct** that rewrites complex English at **CEFR A2**
(elementary) level while preserving the source's facts. Trained on Apple Silicon
with MLX (LoRA), via SFT followed by RL (GSPO) against a decomposed, mostly-deterministic
reward.

> **Format note:** this is an **MLX** model (converted base:
> [`mlx-community/gemma-3-4b-it-bf16`](https://huggingface.co/mlx-community/gemma-3-4b-it-bf16)).
> It loads with `mlx_lm` on Apple Silicon. It is **not** a `transformers`/PyTorch
> checkpoint. The repo also ships the LoRA adapter under [`adapter/`](./adapter)
> for applying on top of the base yourself.

## Usage (MLX)

The model expects its **chat template** with the training **system prompt** — a raw
prompt string will make it ramble. Replicate the training invocation:

```python
from mlx_lm import load, generate

model, tok = load("miguelconner4/claro")

SYSTEM = ("Rewrite the user's text in CEFR A2 (Elementary English): short simple "
          "sentences, basic vocabulary, no idioms. Keep all important facts. "
          "Output only the rewritten text.")

complex_text = "The edifice, constructed circa 1750, was subsequently designated a historic landmark."
prompt = tok.apply_chat_template(
    [{"role": "system", "content": SYSTEM},
     {"role": "user", "content": complex_text}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=512, verbose=False))
# -> "The building was built around 1750. People decided it was important history."
```

## How it was trained

1. **SFT** on ~1,500 (complex → A2) paragraph pairs distilled from a frontier model
   over random Wikipedia paragraphs, filtered by LLM judges.
2. **RL** (200 iters, group size 8, GSPO sequence-level importance sampling,
   KL β=0.1) from the SFT checkpoint, against a **cardinal multiplicative reward**:

   `reward = level_band × vocab × fidelity × format_gates` (each ∈ [0,1]).

   - `level_band` — deterministic A2 difficulty: readability (Flesch), mean sentence
     length, **passive** and **subordination** density, with bands calibrated to the
     10th–90th percentiles of real A2 reference texts.
   - `vocab` — penalty for off-A2-list words, with gloss-aware exemption (defining a
     hard term in-line is not penalized).
   - `fidelity` — LLM judge, decomposed into fact-level **recall** + **hallucination**
     counts (not a holistic score).
   - `format_gates` — hard pass/fail for markdown / degenerate loops.

## Evaluation (30 held-out Wikipedia paragraphs)

  On 30 held-out Wikipedia paragraphs, Claro's rewrites land at **~70% CEFR A2** (most of the rest A1; only ~7% drift up to the harder B1), per a DeepSeek mode-of-3 CEFR classifier — reliably simpler than the model it was fine-tuned from, with fewer too-hard outputs. Faithfulness is preserved: source-fact recall stays ~0.98, and a strict hand-audit (counting only real contradictions and fabricated facts, notparaphrase or omission) found ~3–4 genuine errors across the 30 paragraphs —indistinguishable from the baseline and consistent across three independent judge families (Haiku, GPT-4o, Gemini). So the GSPO step delivered a real gain in simplicity at no measurable cost to accuracy. The few remaining errors are subtle — dropped qualifiers or reversed relations (e.g. "younger"→"older sister").

## Limitations

- **MLX-only** (Apple Silicon). No PyTorch/transformers weights provided.
- Evaluated at n=30; CEFR classification is genuinely noisy at the A2/B1 boundary (judges agree with a strict reference only ~50% of the time there). Treat the numbers as ±10pp.
- ~1 in 10 outputs carries a **genuine fidelity slip** (≈3–4 per 30 in our audit). The dominant mode is subtle attribute/relation errors (e.g. "younger"↔"older sister", a dropped qualifier), not wholesale fabrication.
- English-only; tuned on encyclopedic prose. Out-of-domain text (dialogue, code, poetry) is untested.

## License

Derivative of Google's **Gemma 3**; use is governed by the
[Gemma Terms of Use](https://ai.google.dev/gemma/terms) and the
[Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy),
which carry over to this model.