---
license: mit
library_name: keras-nlp
tags:
  - ielts
  - automated-essay-scoring
  - deberta-v3
  - ordinal-regression
  - evalguide
---

# EvalGuide IELTS AES v2.5

DeBERTa-v3-base ordinal regression model for IELTS Writing Task 2 scoring across four criteria:

- Task Response
- Coherence and Cohesion
- Lexical Resource
- Grammatical Range and Accuracy

## Production checkpoint (current)

| Field | Value |
|-------|-------|
| Variant | **Augmented + calibrated** |
| Weights | `ielts_v2.5_base_en_10ep.weights.h5` |
| Calibration | `ielts_v2.5_base_en_10ep_calibration.pkl` |
| Backbone | `deberta_v3_base_en` |
| Input format | Essay body only (`full_text`) — no question prefix |
| Gold harness QWK | 0.7989 calibrated / 0.8505 raw (1,952-essay holdout) |

### Why this checkpoint is served

1. **Calibrated serving** — Isotonic calibration plus bias correction improves mean-score alignment (SMD −0.07 vs v2.4 +0.08) and lowers RMSE, which matters more for production UX than the higher raw QWK ablation.
2. **Augmented training** — Synonym augmentation (10% of train essays) is part of the documented v2.5 strategy and was verified active in the final run. The no-aug ablation checkpoint is preserved in repo history (first commit).

## Files

| File | Description |
|------|-------------|
| `ielts_v2.5_base_en_10ep.weights.h5` | Model weights (~3.5 GB) |
| `ielts_v2.5_base_en_10ep_calibration.pkl` | Isotonic calibration layer |
| `ielts_v2.5_base_en_10ep_config.json` | Training metadata and metrics |
| `model_config.json` | Production serving config for EvalGuide backend |

## Download

```bash
hf download koecheup/evalguide-ielts-v2.5 --local-dir backend/model
```

Place artifacts under `evalguide_client/backend/model/` alongside `model_config.json`.

## Inference notes

- Tokenize **essay content only**. Do not prepend `Question: …` — training and offline eval use essay-only input.
- Apply the calibration artifact after forward pass when serving the production config.
- Rollback to v2.4: set `IELTS_MODEL_NAME=ielts_v2.4_base_en_10ep.weights.h5`.

## Training summary

- Real data: 9,760 cleaned essays (`ielts_cleaned.csv`)
- Synthetic mix: 15% from 284 cleaned Task 2 essays (`koecheup/ielts-synthetic`)
- Augmentation: 10% synonym replacement (780 train essays)
- Epochs: 10, batch size 8, variance target 2.0 → 2.7

See `docs/backend/v2.5_upgrade_report.md` in the EvalGuide repo for full evaluation tables.