README.md

---
license: cc-by-sa-4.0
base_model: ixa-ehu/roberta-eus-euscrawl-base-cased
language: eu
pipeline_tag: token-classification
tags:
  - grammatical-error-correction
  - gector
  - basque
  - euskara
  - seq2edit
  - error-type-classification
  - multitask
library_name: transformers
---

# GECToR v2 — Multi-task Basque Grammatical Error Correction

A **multi-task GECToR** (Tag, Not Rewrite) model for Basque grammatical error
correction, fine-tuned on [**horkonpon-corpus**](https://github.com/itzune/horkonpon-corpus)
(208K EBE-grounded error pairs). This is the successor to
[`gector-eus`](https://github.com/itzune/gector-eus) (v1), with three key
improvements:

1. **Multi-task architecture** — a third "type" head classifies each edited
   token into one of 8 error categories (spelling, morphology, punctuation,
   capitalization, zalantza, proper_noun, word_level, calque), enabling
   **explainable** corrections for linting/UI use.
2. **EBE-grounded training data** — horkonpon-corpus covers 8+ error categories
   (not just morphology like v1), including real-word errors (zalantza) and
   calques from Spanish/French.
3. **Commercial license** — training data is CC-BY-SA / CC-BY / public-domain,
   so the weights carry **no NonCommercial restriction** (unlike v1).

## Model details

| | |
|---|---|
| **Architecture** | GECToR (RoBERTa-eus-base encoder + 3 heads: edit-label, detect, type) |
| **Base model** | [`ixa-ehu/roberta-eus-euscrawl-base-cased`](https://huggingface.co/ixa-ehu/roberta-eus-euscrawl-base-cased) (Apache 2.0) |
| **Parameters** | ~124M |
| **Training data** | horkonpon-corpus: 161,670 pairs (error + clean natures) |
| **License** | CC-BY-SA 4.0 |
| **Error types** | `none, spelling, punctuation, capitalization, word_level, zalantza, morphology, proper_noun, calque` |

## Evaluation results

Evaluated on the horkonpon-corpus held-out eval split (1,037 errorful +
1,037 clean sentences), `min_error_prob=0.5`, 5 iterations.

### Correction

| Metric | GECToR v2 (mt) | GECToR v1 | Gemma 4 (FT) | Gemma 4 (base) |
|--------|:--------------:|:---------:|:-----------:|:--------------:|
| **F0.5** | 77.6 | 47.5¹ | **80.8** | 2.2 |
| Exact match | 51.3% | 18.0%¹ | 65.5% | 7.0% |
| Precision | 87.6% | 83.8%¹ | 86.3% | 1.8% |
| Recall | 53.3% | 17.4%¹ | 64.4% | 16.5% |
| Clean FP | 1.8% | 1.7%¹ | 8.6% | 97.2% |

¹ v1 on horkonpon eval (domain shift — v1 was trained on Elhuyar morphology
only, so it misses spelling/calque/zalantza errors). v1 scores F0.5=90.2 on its
own Elhuyar Dem eval set.

**Trade-off:** The multi-task type head adds a small cost to correction
(-1.2 F0.5 vs the single-task v2's 78.8) but enables per-token error-type
classification. GECToR wins decisively on **clean FP rate** (1.8% vs Gemma 4's
8.6%) and is lightweight enough for **in-browser ONNX deployment** (~80MB).

The **base Gemma 4 ablation** (F0.5=2.2, 97.2% clean FP) confirms that
fine-tuning is essential — the base LLM has linguistic knowledge but no
minimal-edit discipline. See
[`itzune/gemma-4-e4b-horkonpon`](https://huggingface.co/itzune/gemma-4-e4b-horkonpon)
for the full ablation report.

### Error-type classification (new capability)

| Metric | Value |
|--------|-------|
| Overall word accuracy | 98.1% (15,348 words) |
| **Type accuracy on error words** | **75.8%** (1,115 error words) |

| Category | Precision | Recall | F0.5 | Support |
|----------|-----------|--------|------|---------|
| morphology | 100.0% | 93.5% | **98.6%** | 292 |
| spelling | 95.7% | 92.4% | **95.0%** | 170 |
| punctuation | 97.8% | 70.7% | 90.9% | 258 |
| word_level | 87.8% | 100.0% | 90.0% | 36 |
| zalantza | 82.0% | 85.0% | 82.6% | 89 |
| capitalization | 78.6% | 100.0% | 82.1% | 33 |
| proper_noun | 100.0% | 38.7% | 75.9% | 83 |
| calque | 100.0% | 28.7% | 66.8% | 76 |

**High precision across all categories** — when the model flags a type, it's
almost always correct. Recall is lower for calque and proper_noun because the
detection head often doesn't flag them as errors in the first place (the type
head only fires on detected edits).

## Usage

This is a **GECToR-format model** (custom architecture, not standard
`AutoModel`). Use with the [gotutiyan/gector](https://github.com/gotutiyan/gector)
package (MIT) or the multi-task fork in
[`gector-eus-v2/gector_multitask`](https://github.com/itzune/gector-eus-v2).

```python
import torch
from transformers import AutoTokenizer
from gector import GECToR
from gector.predict import predict, load_verb_dict

# Load model + tokenizer
model = GECToR.from_pretrained("itzune/gector-eus-v2")
model.eval()
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

tokenizer = AutoTokenizer.from_pretrained("itzune/gector-eus-v2", add_prefix_space=True)
tokenizer.add_special_tokens({"additional_special_tokens": ["$START"]})

# Verb-form dictionary (for morphological reinflection)
# Download from gector-eus-v2 repo: data/verb-form-vocab.txt
encode, decode = load_verb_dict("verb-form-vocab.txt")

# Correct a sentence (pre-tokenized: punctuation split from words)
sentences = ["Ni uste hiru hilabetez ohean egoteak eragin zidala ."]
pred_lines, pred_types = predict(
    model, tokenizer, sentences, encode, decode,
    min_error_prob=0.5, batch_size=128, n_iteration=5,
    return_types=True,  # enable error-type head
)

print(pred_lines[0])
# → "Nik uste hiru hilabetez ohean egoteak eragin zidala ."

print(pred_types[0])
# → ['none', 'morphology', 'none', ...]  (per-source-word type labels)
```

### In-browser deployment (ONNX)

Export to int4 ONNX (~80MB) for Transformers.js / WASM deployment:

```bash
PYTHONPATH=src python scripts/export_onnx.py \
    --model_dir itzune/gector-eus-v2 --out_dir onnx
```

The ONNX model exposes three outputs: `logits` (edit labels), `logits_d`
(detection), and `logits_t` (error types) — enabling explainable in-browser
GEC with error-type annotations.

## Training details

| Hyperparameter | Value |
|----------------|-------|
| Epochs | 9 (early-stopped; best at epoch 8) |
| Cold epochs | 2 (encoder frozen, lr=1e-3) |
| Main lr | 1e-5 (linear schedule, 500 warmup) |
| Batch size | 32 |
| Max length | 128 tokens |
| Type loss weight | 0.5 |
| Early stopping | patience=3 (on valid loss) |
| Seed | 42 |
| Hardware | NVIDIA L40 (46GB) |

**Training data composition** (horkonpon-corpus, `error` + `clean` natures):
161,670 pairs including 10,605 mined morphology records. Categories: morphology
(synthetic + mined), spelling, capitalization, punctuation, zalantza
(real-word errors), word_level (h-dropping, sibilant confusion), calque,
proper_noun.

## Limitations

- **Calque recall is low** (28.7%) — literal translations from Spanish/French
  require semantic understanding that the encoder struggles with.
- **Proper noun recall is low** (38.7%) — the model can't know correct forms
  of place names, person names, etc. without world knowledge.
- **Punctuation recall is moderate** (70.7%) — missing commas require
  clause-boundary understanding.
- **Domain-specific** — trained on Basque web text (Berria, Wikipedia, EBE
  examples). Performance may vary on other text domains.
- **Pre-tokenization required** — input must be pre-tokenized (punctuation
  split from words) for the model to work correctly.

## Ethics

This model corrects grammatical errors in Basque text. It should not be used
to penalize or shame language learners, dialect speakers, or writers using
regional Basque variants. The model follows
[Euskaltzaindiaren Euskara Baturaren Eskuliburua (EBE)](https://www.euskaltzaindia.eus/)
as the sole normative authority.

## Citation

```bibtex
@misc{ezpeleta2026gectoreusv2,
  author       = {Ezpeleta, Xabi},
  title        = {GECToR v2: Multi-task Basque Grammatical Error Correction},
  year         = {2026},
  howpublished = {Hugging Face model},
  url          = {https://huggingface.co/itzune/gector-eus-v2}
}
```

### Base model

```bibtex
@misc{artetxe2022euscrawl,
  title         = {Does corpus quality really matter for low-resource languages?},
  author        = {Artetxe, Mikel and Aldabe, Itziar and Agerri, Rodrigo and
                   Perez-de-Viñaspre, Olatz and Soroa, Aitor},
  year          = {2022}, eprint = {2203.08111},
  archivePrefix = {arXiv}, primaryClass = {cs.CL}
}
```

### Architecture

```bibtex
@inproceedings{omelianchuk2020gector,
  title     = {GECToR--Grammatical Error Correction: Tag, Not Rewrite},
  author    = {Omelianchuk, Kostiantyn and Atrasevych, Vitaly and
               Chernodub, Artem and Skurzhanskyi, Oleksandr},
  booktitle = {Proceedings of the Fifteenth Workshop on Innovative Use of
               NLP for Building Educational Applications}, year = {2020}
}
```

## Related

- [horkonpon-corpus](https://github.com/itzune/horkonpon-corpus) — training data
- [gector-eus (v1)](https://github.com/itzune/gector-eus) — predecessor (Elhuyar, NC license)
- [gemma-4-e4b-horkonpon](https://huggingface.co/itzune/gemma-4-e4b-horkonpon) — LLM-based Basque GEC (F0.5=80.8)
- [gotutiyan/gector](https://github.com/gotutiyan/gector) — PyTorch GECToR implementation (MIT)
- [Euskaltzaindia EBE](https://www.euskaltzaindia.eus/) — normative authority