Model Card for HungLe

HungLe is a fine-tuned Large Language Model optimized for Question Answering. It has been trained using QLoRA (Quantized Low-Rank Adaptation) on top of the causal language model Qwen/Qwen2.5-0.5B to achieve high efficiency and low memory footprint.

Model Details

• Developed by: MrHungLe01
• Model type: Causal Language Model (Transformer-based)
• Language(s) (NLP): Vietnamese / English
• License: Apache 2.0
• Finetuned from model: Qwen/Qwen2.5-0.5B

Training Dataset

The model was fine-tuned on a custom dataset managed under the @MrHungLe01 organization:

• Dataset Name: MrHungLe01/my-custom-dataset
• Description: Instruction-following text dataset

Training Hyperparameters (QLoRA)

• Training Method: QLoRA (4-bit quantization)
• Base Model Precision: 4-bit NormalFloat (NF4)
• LoRA Rank (r): 16
• LoRA Alpha (alpha): 32
• LoRA Target Modules: 'q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj' * Learning Rate: 2e-4
• Batch Size: 8
• Epochs: N/A (Trained for 50 steps)

Evaluation & Results

Benchmark Comparison

Performance comparison observed during the project milestones (Full vs LoRA vs QLoRA):

Training Method	Training Time	VRAM Usage	Evaluation Loss
Full Fine-Tuning	[196.24s]	10.18GB	[2.041463]
LoRA (16-bit)	[228.91s]	2.88GB	[2.084870]
QLoRA (HungLe)	[127.93s]	1.7GB	[2.208373]

How to Use

You can easily load and run this model using the Hugging Face transformers library:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MrHungLe01/HungLe"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

prompt = "<|im_start|>user\nXin chào, bạn là ai?<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.7)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))