Qwen-PRIME-4B-AI (LoRA Adapters) 🚀

This repository contains the fine-tuned LoRA (Low-Rank Adaptation) weights for Qwen-PRIME-4B-AI.

These adapters were trained on top of the Qwen/Qwen3-4B-Instruct-2507 base model[cite: 1] to dramatically enhance domain-specific coding capabilities, syntax precision, and structural logic formatting. The fine-tuning process was heavily optimized for consumer hardware, specifically targeting local deployment on constrained VRAM environments (e.g., 4GB NVIDIA GTX 1650)[cite: 1].

📂 Included Files

This adapter repository contains the complete set of weights and configurations required to mount the model[cite: 7]:

• adapter_model.safetensors: The trained LoRA weights[cite: 7].
• adapter_config.json: The PEFT architectural configuration[cite: 7].
• tokenizer.json & tokenizer_config.json: The customized vocabulary mappings[cite: 7].
• chat_template.jinja: The instruction-following formatting rules[cite: 7].

📊 Evaluation Metrics

The fine-tuned model was evaluated quantitatively against the raw base model using exact-match and phrase-match coding datasets. The results demonstrate significant improvements in reproducing target domain syntax:

• ROUGE-1 (Keywords & Terminology): 30.64% (+9.26% over base model)[cite: 5]
• ROUGE-2 (Phrases & Multi-word Logic): 6.29% (+3.43% over base model)[cite: 5]
• ROUGE-L (Sentence & Code Structure): 18.43% (+8.51% over base model)[cite: 5]

🛠️ Training Details

• Base Model: Qwen/Qwen3-4B-Instruct-2507[cite: 1]
• Methodology: Parameter-Efficient Fine-Tuning (PEFT) using QLoRA (4-bit quantization)[cite: 1]
• Framework: Unsloth / Hugging Face Transformers / TRL[cite: 1]
• Compute: Trained via Google Colab (T4 GPU)

💻 Usage (Python / Hugging Face Transformers)

To use these adapters directly in Python, you can load them dynamically on top of the base model using the peft library:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen3-4B-Instruct-2507"
adapter_id = "primeyyy/Qwen-PRIME-4B-AI"

# 1. Load the base model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

# 2. Apply the LoRA adapters
model = PeftModel.from_pretrained(base_model, adapter_id)

# 3. Generate Code
messages = [{"role": "user", "content": "Write a Python script to..."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))