Qwen3-4B-Cybersecurity

Qwen3-4B fine-tuned on 1.28M cybersecurity samples

---

🔪 Abliterated (refusal-removed) version: DexopT/Qwen3-4B-Cybersecurity-Heretic-16bit

Model Description

Qwen3-4B-Cybersecurity is a fine-tuned version of Qwen3-4B-Instruct-2507 specialized in cybersecurity topics including offensive security, penetration testing, vulnerability analysis, malware analysis, and threat intelligence.

Trained with Unsloth SFT on a curated dataset of 1,284,369 samples covering:

• 🔴 Offensive Security — SQL injection, XSS, buffer overflows, reverse shells
• 🔵 Defensive Security — CVE analysis, incident response, hardening
• 🟡 Network Security — MITM, ARP spoofing, DNS poisoning, DDoS
• 🟢 Active Directory — Kerberoasting, Pass-the-Hash, Golden Ticket attacks
• ⚫ Malware Analysis — Ransomware, rootkits, persistence techniques
• 🟣 Web Security — SSRF, XXE, CSRF, LFI/RFI, deserialization

---

Model Family

Model	Description	Link
Qwen3-4B-Cybersecurity	Base fine-tuned model (this repo)	📍 You are here
Qwen3-4B-Cybersecurity-Heretic-16bit	Abliterated — refusal directions removed	→
Qwen3-4B-Cybersecurity-GGUF	Q8_0 + Q4_K_M quantized for llama.cpp	→

---

Training Details

Parameter	Value
Base model	Qwen3-4B-Instruct-2507
Training framework	Unsloth + TRL SFTTrainer
Training samples	1,284,369
Training steps	800
LoRA rank	16
Context length	2048
Batch size	2 × 4 = 8 (effective)
Learning rate	2e-4
LR scheduler	Cosine
Optimizer	AdamW 8bit
Format	16bit merged safetensors
Platform	Google Colab T4

---

Datasets

Dataset	Samples	Description
DexopT/cyber_heretic	1,284,369	Raw unified cybersecurity dataset (JSONL)
DexopT/cyber_heretic_tokenized	1,284,369	Pre-tokenized Arrow dataset (1024 ctx)

Dataset sources include .jsonl, .json, .csv, .parquet, and .arrow formats from multiple cybersecurity repositories, converted to unified OpenAI messages format.

---

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "DexopT/Qwen3-4B-Cybersecurity",
    torch_dtype=torch.float16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("DexopT/Qwen3-4B-Cybersecurity")

messages = [
    {"role": "system", "content": "You are an expert cybersecurity assistant."},
    {"role": "user", "content": "Explain how SQL injection works and how to prevent it."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.8)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

MLX (Apple Silicon)

pip install mlx-lm

mlx_lm.convert \
  --hf-path DexopT/Qwen3-4B-Cybersecurity \
  --mlx-path ~/models/qwen3-cyber-mlx \
  --quantize --q-bits 8

mlx_lm.chat --model ~/models/qwen3-cyber-mlx

LM Studio / llama.cpp / Ollama

Use the GGUF version: DexopT/Qwen3-4B-Cybersecurity-GGUF

---

Refusal Behavior

This model retains some of the original refusal behavior from the base Qwen3 model. For a version with refusal directions removed via Heretic abliteration, see:

🔪 DexopT/Qwen3-4B-Cybersecurity-Heretic-16bit

---

⚠️ Disclaimer

This model is intended for educational and research purposes only. Use responsibly and only on systems you have explicit permission to test. The authors are not responsible for any misuse.

---

Links

🔪 Heretic Version	DexopT/Qwen3-4B-Cybersecurity-Heretic-16bit
📦 GGUF (Q8 + Q4)	DexopT/Qwen3-4B-Cybersecurity-GGUF
📊 Training Dataset	DexopT/cyber_heretic
🔧 Base Model	unsloth/Qwen3-4B-Instruct-2507
🏠 Qwen3 Collection	Qwen3 on HuggingFace