README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-2B
library_name: peft
pipeline_tag: text-generation
language:
  - en
tags:
  - lora
  - peft
  - qwen
  - guardrails
  - code-detection
  - language-identification
  - multi-label-classification
  - quantization
  - 8-bit
metrics:
  - accuracy
  - f1
  - precision
  - recall
model-index:
  - name: CodeLanguage-Qwen3.5-2B-v5
    results:
      - task:
          type: text-classification
          name: Multi-label Programming Language Identification
        dataset:
          name: LangID Guard Held-out Test Set
          type: custom
        metrics:
          - type: accuracy
            name: is_valid accuracy
            value: 1.0000
          - type: accuracy
            name: language-set exact match
            value: 0.9600
          - type: f1
            name: binary F1 (positive=contains code)
            value: 1.0000
          - type: f1
            name: macro F1 over languages
            value: 0.9696
          - type: precision
            name: binary precision (positive=contains code)
            value: 1.0000
          - type: recall
            name: binary recall (positive=contains code)
            value: 1.0000
---
# CodeLanguage-Qwen3.5-2B-v5
LoRA adapter for **Qwen/Qwen3.5-2B** that identifies which programming languages are embedded in a user prompt across **25 languages and configuration formats**. Trained on a combined dataset of Rosetta Code snippets and curated config-language samples (Dockerfile, YAML, Terraform, Makefile, SQL).
The model is fine-tuned to emit a strict JSON object describing the languages found:

```json
{"is_valid": true, "category": {"Python": true, "Bash": true}}
```

`is_valid` is `true` when at least one code/config snippet is present and `false` for natural-language-only prompts. `category` contains only the detected languages, each mapped to `true`; if no code is present `category` is `{}`.
## Quick start
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch, json, re

BASE = "Qwen/Qwen3.5-2B"
ADAPTER = "Accuknoxtechnologies/PromptInjection-Qwen3.5-2B-v5"

SYSTEM_MSG = """You are a code language identifier. For the given user prompt, decide whether it contains any embedded source code (program source or recognizable code-like configuration). Output exactly one JSON object and nothing else: {"is_valid": <true|false>, "category": {"<Lang>": true, ...}}.
No preamble. No explanation. No <think> tags. No markdown code fences. No trailing prose.
Rules:
  - is_valid is TRUE when the prompt contains at least one code/config snippet, FALSE when the prompt is plain natural-language only.
  - category contains ONLY the languages that appear, each mapped to true. If no code is present, category is the empty object {}.
  - When multiple languages appear, list every distinct one (still only true).
Allowed language keys (use these exact spellings):
  Python, JavaScript, Java, C, C++, C#, Go, Rust, Kotlin, Swift, Ruby, R, Scala, Perl, Lua, Bash, PowerShell, Batch, SQL, Dockerfile, YAML, Makefile, Terraform, AWK, jq

Examples:

Input: What's the weather forecast today?
Output: {"is_valid": false, "category": {}}

Input: Run this for me: print('hello world')
Output: {"is_valid": true, "category": {"Python": true}}

Input: Compare these — SELECT * FROM users vs the snippet: console.log(users)
Output: {"is_valid": true, "category": {"SQL": true, "JavaScript": true}}"""

tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    BASE, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, ADAPTER); model.eval()

def langid(prompt: str) -> dict:
    chat = tokenizer.apply_chat_template(
        [{"role":"system","content":SYSTEM_MSG},
         {"role":"user","content":prompt}],
        tokenize=False, add_generation_prompt=True, enable_thinking=False)
    inputs = tokenizer(chat, return_tensors="pt").to(model.device)
    out = model.generate(**inputs, max_new_tokens=220, do_sample=False)
    text = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    return json.loads(re.search(r'\{.*\}', text, re.DOTALL).group(0))
```

## System prompt
The model was trained with the exact system prompt below. Pass it verbatim at inference time — the output schema depends on this prompt.

```text
You are a code language identifier. For the given user prompt, decide whether it contains any embedded source code (program source or recognizable code-like configuration). Output exactly one JSON object and nothing else: {"is_valid": <true|false>, "category": {"<Lang>": true, ...}}.
No preamble. No explanation. No <think> tags. No markdown code fences. No trailing prose.
Rules:
  - is_valid is TRUE when the prompt contains at least one code/config snippet, FALSE when the prompt is plain natural-language only.
  - category contains ONLY the languages that appear, each mapped to true. If no code is present, category is the empty object {}.
  - When multiple languages appear, list every distinct one (still only true).
Allowed language keys (use these exact spellings):
  Python, JavaScript, Java, C, C++, C#, Go, Rust, Kotlin, Swift, Ruby, R, Scala, Perl, Lua, Bash, PowerShell, Batch, SQL, Dockerfile, YAML, Makefile, Terraform, AWK, jq

Examples:

Input: What's the weather forecast today?
Output: {"is_valid": false, "category": {}}

Input: Run this for me: print('hello world')
Output: {"is_valid": true, "category": {"Python": true}}

Input: Compare these — SELECT * FROM users vs the snippet: console.log(users)
Output: {"is_valid": true, "category": {"SQL": true, "JavaScript": true}}
```
## Evaluation
Evaluated on **200 held-out prompts** drawn from `test_dataset_langid.csv` (same single + multi + benign composition as training).

- Evaluation timestamp: `2026-05-22 00:42 UTC`
- GPU: `NVIDIA A10G`
- Source adapter: `Accuknoxtechnologies/PromptInjection-Qwen3.5-2B-v5`
- JSON parse errors: `0/200` (`0.0%`)
### Top-level metrics
| Metric | Value |
|---|---:|
| `is_valid` accuracy | **1.0000** |
| Language-set exact match | **0.9600** |
| Binary F1 (positive = contains code) | **1.0000** |
| Binary precision | 1.0000 |
| Binary recall | 1.0000 |
| Macro F1 across languages | **0.9696** |
### Confusion matrix — binary `is_valid` decision
Positive class = the prompt **contains code** (`is_valid=True`).

| | predicted contains-code | predicted no-code |
|---|---:|---:|
| **actual contains-code** | TP = 181 | FN = 0 |
| **actual no-code**       | FP = 0 | TN = 19 |
### Per-language metrics
Only languages that appear in either the actual or predicted labels are listed.

| Language | support | precision | recall | F1 |
|---|---:|---:|---:|---:|
| `Python` | 14 | 1.000 | 1.000 | 1.000 |
| `Terraform` | 14 | 1.000 | 1.000 | 1.000 |
| `Java` | 12 | 1.000 | 1.000 | 1.000 |
| `C` | 12 | 1.000 | 1.000 | 1.000 |
| `Rust` | 12 | 1.000 | 1.000 | 1.000 |
| `AWK` | 12 | 1.000 | 0.917 | 0.957 |
| `Ruby` | 11 | 0.917 | 1.000 | 0.957 |
| `R` | 11 | 1.000 | 1.000 | 1.000 |
| `Go` | 10 | 1.000 | 0.900 | 0.947 |
| `Swift` | 10 | 1.000 | 0.900 | 0.947 |
| `Scala` | 10 | 1.000 | 0.800 | 0.889 |
| `SQL` | 10 | 1.000 | 1.000 | 1.000 |
| `jq` | 10 | 0.909 | 1.000 | 0.952 |
| `JavaScript` | 9 | 0.900 | 1.000 | 0.947 |
| `Kotlin` | 9 | 1.000 | 1.000 | 1.000 |
| `Perl` | 9 | 1.000 | 1.000 | 1.000 |
| `PowerShell` | 9 | 1.000 | 1.000 | 1.000 |
| `Batch` | 9 | 1.000 | 1.000 | 1.000 |
| `YAML` | 9 | 1.000 | 0.889 | 0.941 |
| `C++` | 7 | 1.000 | 0.857 | 0.923 |
| `C#` | 7 | 0.875 | 1.000 | 0.933 |
| `Lua` | 7 | 1.000 | 0.857 | 0.923 |
| `Bash` | 7 | 1.000 | 1.000 | 1.000 |
| `Dockerfile` | 6 | 0.857 | 1.000 | 0.923 |
| `Makefile` | 6 | 1.000 | 1.000 | 1.000 |

### Inference latency
- Mean: **0.99 s/prompt**
- Median: 0.94 s/prompt
- p95: 1.35 s/prompt
- Max: 1.63 s/prompt

## Training setup
- Base model: `Qwen/Qwen3.5-2B` (loaded in full precision (bf16 / fp16, no `bitsandbytes` quantization))
- LoRA: r=16, alpha=32, dropout=0.05, target modules = {q,k,v,o,gate,up,down}_proj
- Optimizer: adamw_torch, lr=1e-4, cosine schedule, warmup 5%
- Precision: bf16 if available, else fp16
- Effective batch size: 8 (per-device 1 + grad-accum 8), gradient checkpointing on
- Max sequence length: 3200 tokens
- Training data: 10,000 rows  (7,000 single-language + 2,000 multi-language + 1,000 benign)
- Languages: 25 (programming + config formats)

## Supported languages
The model emits one or more of these keys in the `category` map of its JSON output:

```
Python, JavaScript, Java, C, C++, C#, Go, Rust, Kotlin, Swift, Ruby, R, Scala, Perl, Lua, Bash, PowerShell, Batch, SQL, Dockerfile, YAML, Makefile, Terraform, AWK, jq
```

---
*Model card generated automatically by `eval_and_push_card.py` on 2026-05-22 00:42 UTC.*