Nitinsaini077/SainiON-BugHunt

SainiON Hacks Bug Hunting

A fine-tuned Qwen 3B language model optimized for improved conversational responses using LoRA (Low-Rank Adaptation). The LoRA adapter has been merged into the base model, making this a standalone model ready for inference.

---

Base Model

• Architecture: Qwen 3B
• Fine-tuning Method: LoRA
• Merged: Yes
• Framework: Transformers
• Format: Hugging Face Transformers

---

Features

• Improved conversational responses
• Better greeting and small-talk handling
• Improved instruction following
• Friendly assistant behavior
• Standalone merged model (no LoRA adapter required)

---

Training Details

Item	Value
Method	LoRA Fine-tuning
Epochs	3
Optimizer	AdamW
Learning Rate	2e-4
Batch Size	2
Gradient Accumulation	4

---

Intended Use

This model is intended for:

• General conversation
• Question answering
• Writing assistance
• Programming assistance
• Educational purposes
• Research and experimentation

---

Limitations

• Knowledge is limited to the base model's training cutoff.
• May produce incorrect or hallucinated information.
• Responses should be verified before use in critical environments.
• Not intended for high-risk or safety-critical decisions.

---

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Nitinsaini077/SainiON-BugHunt"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto"
)

---

Example

messages = [
    {"role": "user", "content": "Hi"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=128
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Training Pipeline

1. Loaded the base Qwen 3B model
2. Applied LoRA fine-tuning
3. Trained on conversational instruction-response pairs
4. Merged LoRA weights into the base model
5. Exported as a standalone Hugging Face model

---

Repository Structure

config.json
generation_config.json
model-00001-of-00002.safetensors
model-00002-of-00002.safetensors
model.safetensors.index.json
tokenizer.json
tokenizer_config.json
chat_template.jinja

---

Future Improvements

• Larger instruction datasets
• Cybersecurity and VAPT fine-tuning
• AI Security datasets
• Retrieval-Augmented Generation (RAG)
• Better reasoning and code generation
• GGUF export for local inference

---

License

Please refer to the license of the original Qwen base model. This repository contains a fine-tuned derivative and should be used in accordance with the original model's license.

---

Acknowledgements

• Alibaba Qwen Team
• Hugging Face Transformers
• PEFT (LoRA)
• TRL