Genuine-7B-Instruct

A fine-tuned Qwen 2.5 7B Instruct model, tuned for more engaging conversation with fewer sycofant responses.

Model Details

This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct using the Unsloth framework with LoRA (Low-Rank Adaptation) for efficient training.

• Developed by: theprint
• Model type: Causal Language Model (Fine-tuned with LoRA)
• Language: en
• License: apache-2.0
• Base model: Qwen/Qwen2.5-7B-Instruct
• Fine-tuning method: LoRA with rank 128

Intended Use

Brainstorming, idea development, general conversation

GGUF Quantized Versions

Quantized GGUF versions are available in the theprint/Genuine-7B-Instruct-GGUF repo.

• Genuine-7B-Instruct-f16.gguf (14531.9 MB) - 16-bit float (original precision, largest file)
• Genuine-7B-Instruct-q3_k_m.gguf (3632.0 MB) - 3-bit quantization (medium quality)
• Genuine-7B-Instruct-q4_k_m.gguf (4466.1 MB) - 4-bit quantization (medium, recommended for most use cases)
• Genuine-7B-Instruct-q5_k_m.gguf (5192.6 MB) - 5-bit quantization (medium, good quality)
• Genuine-7B-Instruct-q6_k.gguf (5964.5 MB) - 6-bit quantization (high quality)
• Genuine-7B-Instruct-q8_0.gguf (7723.4 MB) - 8-bit quantization (very high quality)

Training Details

Training Data

This data set was created to limit sycofancy in language models and encouraging the models to (gently) push back and call out bad ideas.

• Dataset: theprint/Gentle-Pushback-8.5k-alpaca
• Format: alpaca

Training Procedure

• Training epochs: 2
• LoRA rank: 128
• Learning rate: 0.0001
• Batch size: 6
• Framework: Unsloth + transformers + PEFT
• Hardware: NVIDIA RTX 5090

Usage

from unsloth import FastLanguageModel
import torch

# Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="theprint/Genuine-7B-Instruct",
    max_seq_length=4096,
    dtype=None,
    load_in_4bit=True,
)

# Enable inference mode
FastLanguageModel.for_inference(model)

# Example usage
inputs = tokenizer(["Your prompt here"], return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Alternative Usage (Standard Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "theprint/Genuine-7B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("theprint/Genuine-7B-Instruct")

# Example usage
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Your question here"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

Using with llama.cpp

# Download a quantized version (q4_k_m recommended for most use cases)
wget https://huggingface.co/theprint/Genuine-7B-Instruct/resolve/main/gguf/Genuine-7B-Instruct-q4_k_m.gguf

# Run with llama.cpp
./llama.cpp/main -m Genuine-7B-Instruct-q4_k_m.gguf -p "Your prompt here" -n 256

Limitations

May provide incorrect information.

Citation

If you use this model, please cite:

@misc{genuine_7b_instruct,
  title={Genuine-7B-Instruct: Fine-tuned Qwen/Qwen2.5-7B-Instruct},
  author={theprint},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/theprint/Genuine-7B-Instruct}
}

Acknowledgments

• Base model: Qwen/Qwen2.5-7B-Instruct
• Training dataset: theprint/Gentle-Pushback-8.5k-alpaca
• Fine-tuning framework: Unsloth
• Quantization: llama.cpp