README.md

---
base_model: Qwen/Qwen3-4B
datasets:
- twnlp/ChineseErrorCorrector
language:
- zh
license: apache-2.0
metrics:
- f1
- precision
- recall
pipeline_tag: text-generation
library_name: transformers
tags:
- chinese
- text-correction
- grammatical-error-correction
- spelling-check
- qwen3
- chain-of-thought
- reinforcement-learning
model-index:
- name: ChineseErrorCorrector4-4B
  results:
  - task:
      type: text-generation
      name: Chinese Grammatical Error Correction
    dataset:
      name: NACGEC
      type: nacgec
    metrics:
    - type: f0.5
      value: 50.99
  - task:
      type: text-generation
      name: Chinese Spelling Check
    dataset:
      name: CSCD
      type: cscd
    metrics:
    - type: f1
      value: 59.61
---

# ChineseErrorCorrector4-4B (CSRP)

<p align="center">
  <a href="https://github.com/TW-NLP/ChineseErrorCorrector">
    <img src="https://img.shields.io/badge/GitHub-Repository-black?logo=github" alt="GitHub"/>
  </a>
  &nbsp;
  <a href="https://huggingface.co/twnlp/ChineseErrorCorrector4-4B">
    <img src="https://img.shields.io/badge/🤗-Model%20Card-yellow" alt="Hugging Face"/>
  </a>
  &nbsp;
  <img src="https://img.shields.io/badge/ACL%202026-Oral-red" alt="ACL 2026 Oral"/>
  &nbsp;
  <img src="https://img.shields.io/badge/License-Apache%202.0-blue" alt="License"/>
</p>

---

**ChineseErrorCorrector4-4B** is a high-precision Chinese Grammatical Error Correction (CGEC) and Chinese Spelling Check (CSC) model, presented in the paper [CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards](https://huggingface.co/papers/2606.00020).

## 🔥 Recent Updates

| Date | Update |
|------|--------|
| **2026-05** | 🎉 Paper **"CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards"** accepted as **Oral** at **ACL 2026** |
| **2026-05** | 🚀 Released **ChineseErrorCorrector4-4B**, achieving new **SOTA** on both NACGEC and CSCD benchmarks |

---

## 💡 Introduction

**ChineseErrorCorrector4-4B** is built on the **CSRP (CPT → SFT → RL)** three-stage training framework.

### The Problem: Over-Correction Bias

Traditional LLM-based correction systems often suffer from **over-correction bias** — models unnecessarily paraphrase correct text rather than leaving it untouched. CSRP resolves this by calibrating decision boundaries through a structured curriculum:

| Stage | Name | Description |
|-------|------|-------------|
| **Phase I** | Balanced Continued Pre-training (CPT) | Internalizes linguistic priors using 5.9M samples with an 8:2 mixture of general and correction-specific data |
| **Phase II** | Rationale-Augmented SFT | Distills Chain-of-Thought reasoning paths to guide the model in diagnosing error types before executing corrections |
| **Phase III** | Efficiency-Aware Policy Alignment | Uses **GRPO** with a novel **Efficiency-Aware Reward (EAR)** to penalize unnecessary edits and reward surgical precision |

---

## 📊 Benchmark Results

### 榜单一：中文语法纠错（CGEC）— NACGEC 基准

针对原生中文及学习者文本，CSRP (4B) 斩获新 SOTA，$F_{0.5}$ 高达 **50.99**，显著超越此前最优专业大模型。

| 模型 (Scale) | 准确率 Precision | 召回率 Recall | **$F_{0.5}$** (核心指标) |
|:-------------|:---:|:---:|:---:|
| BART | 34.67 | 41.88 | 35.91 |
| HW-CGEC | 50.95 | 32.29 | 45.26 |
| ScholarGEC (14B) | 45.08 | 59.33 | 47.35 |
| CEC3 (4B) | 54.20 | 34.75 | 48.74 |
| **CSRP (4B) [Ours]** ✅ | **57.17** | **35.60** | **50.99** |

---

### 榜单二：中文拼写检查（CSC）— CSCD 基准

CSRP 在字符级纠错 F1 上同样展现出强劲统治力，达到惊人的 **59.61**，全面超越 GPT-4。

| 模型 | Correction F1 |
|:-----|:---:|
| BERT | 25.49 |
| SoftMask | 44.48 |
| SMBERT | 44.67 |
| MDCSpell+ARM | 48.93 |
| GPT-4 (Few-shot) | 54.41 |
| **CSRP (4B) [Ours]** ✅ | **59.61** |

---

## 🛠️ Quick Start

### Requirements

```bash
pip install -U transformers torch
```

> **Note:** Requires `transformers >= 4.51.0` for Qwen3 architecture support.

### Inference with Transformers

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "twnlp/ChineseErrorCorrector4-4B"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

# Professional instruction template
instruction = (
    "假如你是一名专业的纠错专家，请分析输入句子的语法错误类型和修改原因，"
    "并只输出纠正后的语句，错误类型如下：错别字、词语搭配错误、词性错误、"
    "语序错误、成分残缺、成分赘余、关联词使用错误、指代不明、语义逻辑不通、无误。"
)

text_input = "下个星期，我跟我朋唷打算去法国玩儿。"

messages = [
    {"role": "system", "content": instruction},
    {"role": "user",   "content": text_input}
]

text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512,
    do_sample=False,
    repetition_penalty=1.1
)

response = tokenizer.decode(
    generated_ids[0][len(model_inputs.input_ids[0]):],
    skip_special_tokens=True
)
print(response)
```

---

## 📝 Output Example

**Input:**
```
下个星期，我跟我朋唷打算去法国玩儿。
```

**Model Output:**
```
<think>
错误类型：错别字
修改原因：原句中的"朋唷"是错误写法，正确应为"朋友"。
"唷"是语气助词，不能用于此处指代同伴。
正确句使用"朋友"准确表达了与说话者一同前往的人，避免了因错别字造成的语义误解。
</think>

下个星期，我跟我朋友打算去法国玩儿。
```

---

## 📜 License

This project is released under the [Apache 2.0 License](LICENSE).

## Citation

如果本工作对您有帮助，欢迎引用：

```bibtex
@misc{tian2026csrpchainofthoughtreasoningchinese,
      title={CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards}, 
      author={Wei Tian and Yuhao Zhou and Man Lan},
      year={2026},
      eprint={2606.00020},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.00020}, 
}
```