GRMR-V3-Q4B

GRMR-V3-Q4B is a fine-tuned version of unsloth/Qwen3-4B-Base specifically optimized for grammar correction tasks.

IMPORTANT: Please ensure you are using the following sampler settings for optimal results:

temperature = 0.7
frequency_penalty = 0.0
presence_penalty = 0.0
min_p = 0.01
top_p = 0.95
top_k = 40

Model description

GRMR-V3-Q4B is a grammar correction model built on Qwen3 4B base model. It has been fine-tuned on a large dataset of grammar correction examples to help improve text quality by fixing grammatical errors, punctuation, spelling, and other language issues.

The model uses a specialized chat template that structures inputs as "text" and outputs as "corrected" to maintain a clear distinction between original and corrected content.

Model Details

Developed by: qingy2024

Finetuned from model: unsloth/Qwen3-4B-Base

Model type: Qwen3

Language(s): English

License: apache-2.0

Training Dataset: qingy2024/grmr-v4-60k

---

Manual Testing Examples

Here are a few examples of grammar corrections this model can handle:

Original Text	Corrected Text
i dont know weather to bring a umbrella today	I don't know whether to bring an umbrella today.
she go to the store yesterday	She went to the store yesterday.
they is going to be late for the meeting	They are going to be late for the meeting.
the cat laying on the floor all day	The cat is laying on the floor all day.

---

Training procedure

The model was fine-tuned using full parameter fine-tuning (not LoRA) on the GRMR-V4-60K dataset. The training was optimized using the Unsloth framework for efficient training of LLMs.

Training hyperparameters

• Batch size: 8
• Gradient accumulation steps: 2
• Learning rate: 5e-5
• Epochs: 1
• Optimizer: AdamW (8-bit)
• Weight decay: 0.01
• LR scheduler: Cosine
• Warmup steps: 180
• Max sequence length: 16,384
• Training precision: Mixed precision (BF16 where available, FP16 otherwise)

Intended uses & limitations

This model is designed for grammar correction tasks. It can be used to:

• Fix grammatical errors in written text
• Correct punctuation
• Address spelling mistakes
• Improve sentence structure and clarity

Limitations

• The model may struggle with highly technical or domain-specific content
• It may not fully understand context-dependent grammar rules in all cases
• Performance may vary for non-standard English or text with multiple errors

How to use

Projects based on Hugging Face transformers should be able to run this model easily.

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load model and tokenizer
model_name = "qingy2024/GRMR-V3-Q4B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Text with grammar errors to correct
text_to_correct = "i am going to the store tommorow and buy some thing for dinner"
# Format as messages
messages = [
    {"role": "user", "content": text_to_correct}
]
# Apply the custom chat template
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=512,
    temperature=0.1,
    do_sample=True
)
# Decode and print the corrected text
corrected_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected_text)

Using with the Hugging Face pipeline

from transformers import pipeline
pipe = pipeline(
    "text-generation",
    model="qingy2024/GRMR-V3-Q4B",
    torch_dtype="auto",
    device_map="auto"
)
messages = [
    {"role": "user", "content": "i dont know weather to bring a umbrella today"}
]
result = pipe(
    messages,
    max_new_tokens=100,
    temperature=0.1,
    do_sample=True,
    return_full_text=False
)[0]["generated_text"]
print(result)

Custom Chat Template

The model uses a custom chat template with special formatting for grammar correction:

• User inputs are formatted with <|text_start|> and <|text_end|> tags
• Model outputs are formatted with <|corrected_start|> and <|corrected_end|> tags

The complete chat template is:

{%- for message in messages %}
{%- if message.role == "user" %}
{{- '<|text_start|>\n' + message.content + '<|text_end|>\n' }}
{%- elif message.role == "assistant" %}
{{- '<|corrected_start|>\n' + message.content + '<|corrected_end|>\n' }}
{%- else %}
{{- raise('Unknown role: ' + message.role) }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|corrected_start|>\n' }}
{%- endif %}

Training Dataset

The model was fine-tuned on the qingy2024/grmr-v4-60k dataset, which contains 60,000 examples of original text and their grammatically corrected versions.

Bias, Risks, and Limitations

• The model may reflect biases present in the training data
• It may not perform equally well across different writing styles or domains
• The model might occasionally introduce errors or change the meaning of text
• It focuses on grammatical correctness rather than stylistic improvements

Citations

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report}, 
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388}, 
}

Contact

For questions or issues related to the model, please reach out via Hugging Face or by creating an issue in the repository.