thenewth/Qwen1.5-0.5B-Chat-GrammarCorrection

Qwen1.5-0.5B Grammar Correction

This model is a fine-tuned version of Qwen/Qwen1.5-0.5B-Chat, optimized specifically for grammar correction tasks.

Model Description

This model improves English grammar by correcting spelling, punctuation, and sentence structure. It was trained using a structured correction prompt format to ensure consistent and predictable outputs.

Model Details

Developed by: thenewth
Base model: Qwen/Qwen1.5-0.5B-Chat
Model type: Qwen 1.5 — Chat
Language(s): English
License: apache-2.0
Training Dataset: thenewth/grmr-v4-60k

---

Manual Testing Examples

Original Text	Corrected Text
i can’t hardly understand what he say	I can't hardly understand what he says.
she don’t likes coffee in the morning	She doesn't like coffee in the morning.
we was going to the park but it start raining	We were going to the park, but it started raining.
he forget to bring his laptop yesterday	He forgot to bring his laptop yesterday.
the dog runned outside when the door open	The dog ran outside when the door opened.

---

Training Procedure

The model was fine-tuned using Hugging Face AutoTrain with LoRA (PEFT) and int4 quantization.

Training Hyperparameters

• Batch size: 8
• Gradient accumulation steps: 2
• Learning rate: 5e-5
• Epochs: 1
• Optimizer: AdamW (8-bit)
• Weight decay: 0.01
• LR scheduler: Cosine
• LoRA rank (r): 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Block size (max sequence length): 16,384
• Mixed precision: FP16
• Quantization: INT4
• Trainer: SFT

How to Use

Below is the recommended prompt structure and example usage code.

Prompt Template

def make_prompt(original, corrected=''):
  prompt = f"""Your task is to correct the user's grammar. Create Corrected Text by correcting errors in the Original Text.
  ### Original Text: {original}
  ### Corrected Text: {corrected}"""
  return prompt

Hugging Face Pipeline Example

from transformers import pipeline
pipe = pipeline(
    "text-generation",
    model="thenewth/Qwen1.5-0.5B-Chat-GrammarCorrection",
    torch_dtype="auto",
    device_map="auto"
)      
example = make_prompt('they is going to be late for the meeting')
messages = [
    {"role": "user", "content": example}
]
result = pipe(
    messages,
    max_new_tokens=100,
    temperature=0.1,
    do_sample=True,
    return_full_text=False
)[0]["generated_text"]
print(result)

Intended Uses & Limitations

This model is designed for general English grammar correction.

Limitations

• May struggle with domain-specific jargon
• May rewrite phrasing to improve clarity
• May occasionally change meaning when structure is unclear

Bias, Risks, and Considerations

• Model reflects biases present in training data
• Performance varies based on writing style and clarity

Contact

For issues or inquiries, please contact via Hugging Face model page:
thenewth/Qwen1.5-0.5B-Chat-GrammarCorrection