Update README.md

README.md

@@ -1,63 +1,265 @@
 ---
+language:
+- gl
+license: cc-by-4.0
 base_model: carvalho-nos/CarvalhoChat_v4
 library_name: peft
-model_name: rq1_carvalhochat_gl_gec_lora
+pipeline_tag: text-generation
 tags:
-- base_model:adapter:carvalho-nos/CarvalhoChat_v4
+- galician
+- grammatical-error-correction
+- gec
+- grammar-correction
+- orthographic-correction
+- text-correction
 - lora
-- sft
-- transformers
-- trl
-- unsloth
-licence: license
-pipeline_tag: text-generation
+- peft
+- carvalho
+- carvalhochat
+- llama
+- low-resource-nlp
+- proxecto-nos
 ---
 
-# Model Card for rq1_carvalhochat_gl_gec_lora
+# CarvalhoChat_GEC
+
+## Model Summary
+
+**CarvalhoChat_GEC** is a LoRA adapter for **Galician grammatical error correction (GEC)**. It is designed to correct Galician sentences while preserving their original meaning and making minimal necessary changes.
+
+The adapter was fine-tuned from **`carvalho-nos/CarvalhoChat_v4`** using high-quality Galician GEC data only. The training setup was designed to evaluate the effect of the initial model checkpoint on Galician GEC performance while keeping the data, hyperparameters, prompt format, inference procedure, and evaluation setup fixed.
+
+This repository contains a **PEFT/LoRA adapter**, not a full merged model. To use it, load the base model first and then apply the adapter.
+
+The adapter was trained with supervised fine-tuning using **Unsloth** for efficient LoRA training. Training was performed on an **NVIDIA A100 80GB GPU** and took approximately **5 hours**.
+
+## Intended Task
+
+The model is intended for **Galician grammatical error correction**.
+
+Given an input sentence in Galician, the model should return only the corrected sentence.
+
+### Example
+
+Input:
+
+```text
+<|task:gec|>
+<|lang:gl|>
+Frase: A nena chegou tarde porque tiña moitos cousas que facer.
+```
+
+Expected output:
+
+```text
+A nena chegou tarde porque tiña moitas cousas que facer.
+```
+
+## Base Model
+
+- **Base model:** `carvalho-nos/CarvalhoChat_v4`
+- **Adapter type:** LoRA / PEFT
+- **Task:** Galician grammatical error correction
+- **Language:** Galician
+
+## Training Data
+
+The adapter was trained exclusively on high-quality Galician GEC data. The training data were drawn from the private dataset repository:
+
+- `proxectonos/galician-gec-corpora`
+
+The training mixture includes:
+
+- **CORTEGAL-derived Galician correction data**
+- **Initial corrected synthetic Galician GEC data**
+- **A 1,500-example ParlaMint punctuation/correction subset**
+
+
+## Prompt Format
+
+The model was trained with a chat-style prompt using the tokenizer chat template of the base model.
+
+The system prompt used during training was:
+
+```text
+Es un corrector gramatical. Corrixe a frase na lingua de destino solicitada. Fai cambios mínimos. Conserva o significado. Devolve só a frase corrixida.
+```
+
+The user message follows this format:
+
+```text
+<|task:gec|>
+<|lang:gl|>
+Frase: {sentence}
+```
 
-This model is a fine-tuned version of [carvalho-nos/CarvalhoChat_v4](https://huggingface.co/carvalho-nos/CarvalhoChat_v4).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+The assistant target is the corrected Galician sentence only.
 
-## Quick start
+## Special Tokens
+
+The adapter uses the following special tokens:
+
+```text
+<|task:gec|>
+<|lang:gl|>
+```
+
+These tokens were added to the tokenizer during training and their embeddings were made trainable together with the LoRA parameters.
+
+For inference, these tokens should be included in the user prompt exactly as shown above.
+
+## LoRA Configuration
+
+The LoRA adapter was trained over the main attention and MLP projection modules typically used in Llama-style architectures:
+
+```text
+q_proj
+k_proj
+v_proj
+o_proj
+gate_proj
+up_proj
+down_proj
+```
+
+Training configuration used in the experimental setup:
+
+```text
+LoRA rank: 16
+LoRA alpha: 32
+LoRA dropout: 0.0
+Max sequence length: 1024
+Learning rate: 2e-4
+Warmup ratio: 0.05
+Weight decay: 0.01
+Scheduler: cosine
+Optimizer: adamw_8bit
+```
+
+The model was trained using 4-bit loading for memory efficiency.
+
+## How to Use
+
+Install the required libraries:
+
+```bash
+pip install -U transformers peft accelerate bitsandbytes torch
+```
+
+Then load the base model and adapter:
 
 ```python
-from transformers import pipeline
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
+from peft import PeftModel
+
+BASE_MODEL = "carvalho-nos/CarvalhoChat_v4"
+ADAPTER = "proxectonos/CarvalhoChat_GEC"
+
+SPECIAL_TOKENS = [
+    "<|task:gec|>",
+    "<|lang:gl|>",
+]
+
+SYSTEM_PROMPT = (
+    "Es un corrector gramatical. "
+    "Corrixe a frase na lingua de destino solicitada. "
+    "Fai cambios mínimos. "
+    "Conserva o significado. "
+    "Devolve só a frase corrixida."
+)
+
+def build_messages(sentence: str):
+    return [
+        {
+            "role": "system",
+            "content": SYSTEM_PROMPT,
+        },
+        {
+            "role": "user",
+            "content": (
+                "<|task:gec|>\n"
+                "<|lang:gl|>\n"
+                f"Frase: {sentence.strip()}"
+            ),
+        },
+    ]
+
+def clean_output(text: str) -> str:
+    for token in SPECIAL_TOKENS:
+        text = text.replace(token, "")
+    return text.strip().splitlines()[0].strip()
+
+quant_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_compute_dtype=torch.float16,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=True,
+)
+
+tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    BASE_MODEL,
+    quantization_config=quant_config,
+    device_map="auto",
+    torch_dtype=torch.float16,
+)
+
+model = PeftModel.from_pretrained(base_model, ADAPTER)
+model.eval()
+
+sentence = "A nena chegou tarde porque tiña moitos cousas que facer."
+messages = build_messages(sentence)
+
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    output_ids = model.generate(
+        **inputs,
+        max_new_tokens=64,
+        do_sample=False,
+        repetition_penalty=1.05,
+        pad_token_id=tokenizer.eos_token_id,
+        eos_token_id=tokenizer.eos_token_id,
+    )
+
+generated = output_ids[0][inputs["input_ids"].shape[1]:]
+prediction = tokenizer.decode(generated, skip_special_tokens=True)
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
+print(clean_output(prediction))
 ```
 
-## Training procedure
+## Expected Output Format
 
- 
+The model is expected to return only the corrected Galician sentence.
 
+It should not provide explanations, comments, lists of changes, or alternative corrections.
 
-This model was trained with SFT.
+## Limitations
 
-### Framework versions
+- The model is specialized for Galician grammatical error correction.
+- It was trained only on high-quality Galician GEC data, so its coverage depends on the error types represented in those datasets.
+- The model may overcorrect valid Galician variants or make unnecessary edits in ambiguous cases.
+- The model should not be used as the only authority for normative linguistic decisions.
+- Human review is recommended for formal, legal, educational, or publication-critical uses.
 
-- PEFT 0.18.1
-- TRL: 0.24.0
-- Transformers: 5.5.0
-- Pytorch: 2.10.0
-- Datasets: 4.3.0
-- Tokenizers: 0.22.2
+## Intended Uses
 
-## Citations
+This adapter can be used for:
 
+- Galician grammatical error correction
+- orthographic and punctuation correction
+- evaluation of Galician GEC systems
+- research on low-resource grammatical correction
+- comparison of base-model adaptation strategies for Galician
 
+## Acknowledgements
 
-Cite TRL as:
-    
-```bibtex
-@misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
-}
-```
+This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA. Esta publicación del proyecto Desarrollo de Modelos ALIA está financiada por el Ministerio para la Transformación Digital y de la Función Pública y por el Plan de Recuperación, Transformación y Resiliencia – Financiado por la Unión Europea – NextGenerationEU.