license: apache-2.0
base_model: unsloth/Qwen3-4B-Instruct-2507
tags:
- unsloth
- lora
- qwen3
- spanish
- rioplatense
- uruguay
- gguf
language:
- es
library_name: peft
pipeline_tag: text-generation
CharruaDevs Qwen3-4B (LoRA + GGUF)
Conversational fine-tune of Qwen3-4B-Instruct-2507 that replies in the style of the Uruguayan subreddit r/CharruaDevs: informal Rioplatense Spanish, voseo, and short, opinionated developer-forum answers.
The goal is not to teach the model new knowledge but to transfer the style and opinions of the community. For that reason the LoRA targets only the attention projections (q_proj, k_proj, v_proj, o_proj) and leaves the MLP blocks untouched, so training changes how the model speaks rather than what it knows. Trained with QLoRA (4-bit, r=32, alpha=64) on roughly 11k real post/comment pairs from the subreddit.
Two variants
This repo ships two GGUF variants of the same project, trained on different data mixes. They trade style intensity for conversational ability:
chat (recommended) |
raw |
|
|---|---|---|
| Training data | single-turn pairs + 1.3k real multi-turn comment chains | single-turn post/comment pairs only |
| Multi-turn chat | works | collapses after the first reply |
| Style intensity (single-shot) | strong | strongest |
| Best for | chatting in Ollama / LM Studio | one-shot forum-style answers |
In a pairwise LLM-judge eval both variants beat the base model on subreddit-style fidelity (chat: 7-1, raw: 6-2), while the raw variant wins head-to-head on single-shot style but returns empty or off-distribution replies from the second turn onwards.
Quick start (Ollama)
Chat variant (recommended):
ollama run hf.co/Jabr7/charruadevs-4b:charruadevs-4b-chat.Q4_K_M.gguf
Raw variant (single-shot only, phrase your message like a forum post and reset the session between questions):
ollama run hf.co/Jabr7/charruadevs-4b:charruadevs-4b-raw.Q4_K_M.gguf
Suggested sampling for both: temperature=0.7, top_p=0.9, repeat_penalty=1.1. The models were trained without a system prompt, so leave the system field empty for best results.
Using the adapter (Python)
The adapter_model.safetensors in this repo is the chat variant.
from unsloth import FastLanguageModel
model, tok = FastLanguageModel.from_pretrained("Jabr7/charruadevs-4b", load_in_4bit=True)
FastLanguageModel.for_inference(model)
msgs = [{"role": "user", "content": "¿Qué opinan de Genexus?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to("cuda")
out = model.generate(ids, max_new_tokens=200, temperature=0.7, top_p=0.9)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
Training details
- Base model:
unsloth/Qwen3-4B-Instruct-2507 - Method: 4-bit QLoRA, target modules
q_proj,k_proj,v_proj,o_proj, r=32, alpha=64 - Loss: computed on the assistant completion only (train on responses)
- chat variant: ~1500 steps on single-turn pairs plus multi-turn chains reconstructed from real Reddit comment trees, with dialect-heavy and high-karma examples oversampled
- raw variant: ~1300 steps on single-turn pairs with dialect-heavy examples oversampled
- Data: real comments from r/CharruaDevs (custom dataset, not included)
Limitations
This is a style and opinion model, not a factual source. It can confidently make up salaries, dates, statistics, and other numbers, so do not trust them. It also reproduces the humor, writing quirks, and biases of the forum. Intended for demonstration and entertainment.