Accuknoxtechnologies/Code-Qwen3.5-2B-LoRA-8bit

PII-Qwen3.5-2B-LoRA-8bit

LoRA adapter for Qwen/Qwen3.5-2B that detects embedded code snippets in user prompts across 10 programming languages (bash, c, go, java, javascript, php, python, ruby, rust, sql) plus a generic Code pattern. Trained on the LLM Guard code and ban_code scanner outputs. The model is fine-tuned to emit a strict JSON object describing every code-snippet location found in the user prompt:

{"is_valid": false, "violations": {"python": [[5, 25]], "sql": [[40, 60]]}}

Quick start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch, json, re

BASE = "Qwen/Qwen3.5-2B"
ADAPTER = "Yash1005/PII-Qwen3.5-2B-LoRA-8bit"

tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
bnb = BitsAndBytesConfig(load_in_8bit=True)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER); model.eval()

def guard(prompt: str) -> dict:
    chat = tokenizer.apply_chat_template(
        [{"role":"system","content":SYSTEM_MSG},
         {"role":"user","content":prompt}],
        tokenize=False, add_generation_prompt=True, enable_thinking=False)
    inputs = tokenizer(chat, return_tensors="pt").to(model.device)
    out = model.generate(**inputs, max_new_tokens=768, do_sample=False)
    text = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    return json.loads(re.search(r'\{.*\}', text, re.DOTALL).group(0))

Evaluation

Evaluated on 100 held-out prompts drawn from test_dataset_code.csv

• Source adapter: Yash1005/Code-Qwen3.5-2B-LoRA-8bit
• JSON parse errors: 0/100 (0.0%)

Top-level metrics

Metric	Value
is_valid accuracy	0.9500
Violation-type-set exact match	0.8100
Binary F1 (positive = invalid)	0.9474
Binary precision	1.0000
Binary recall	0.9000
Macro F1 across violation types	0.7545

Confusion matrix — binary is_valid decision

Positive class = the prompt contains a violation (is_valid=False).

	predicted invalid	predicted valid
actual invalid	TP = 45	FN = 5
actual valid	FP = 0	TN = 50

Per violation-type metrics

Only types that appear in either the actual or predicted labels are listed.

Type	support	precision	recall	F1
python	12	0.917	0.917	0.917
sql	10	1.000	0.500	0.667
bash	8	1.000	0.250	0.400
javascript	8	0.778	0.875	0.824
rust	7	1.000	0.714	0.833
java	6	1.000	0.833	0.909
php	5	1.000	1.000	1.000
ruby	5	1.000	0.600	0.750
Code	5	0.000	0.000	0.000
c	4	1.000	1.000	1.000
go	4	1.000	1.000	1.000

Supported violation types

The model emits one or more of these TYPE keys in the violations map of its JSON output:

bash, c, go, java, javascript, php, python, ruby, rust, sql, Code