README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct
tags:
  - code
  - security
  - secure-coding
  - lora
  - qwen2.5-coder
language:
  - en
pipeline_tag: text-generation
---

# Cipheron

**Cipheron** is a small, LoRA fine-tuned coding model specialized in **secure code review** — given a piece of code, it tries to spot common security vulnerabilities and suggest a fixed, secure version.

- **Base model**: [Qwen/Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) (Apache 2.0)
- **Method**: LoRA fine-tuning (r=16, alpha=32), 3 epochs, ~830 steps
- **Training data**: [CyberNative/Code_Vulnerability_Security_DPO](https://huggingface.co/datasets/CyberNative/Code_Vulnerability_Security_DPO) (~4.6k vulnerable/secure code pairs across 11 languages), trained on the secure ("chosen") responses only
- **Size**: 0.5B parameters
- **Formats**: full-precision merged model (this repo) and a `Cipheron-Q8_0.gguf` quantized file for on-device / CPU / phone use via llama.cpp, Ollama, or similar runners

## What it's good at

In testing, Cipheron reliably identifies and correctly fixes:
- **SQL injection** (rewrites string-concatenated queries as parameterized queries)
- **Command injection** (rewrites `os.system`/shell string concatenation as safer `subprocess` calls)

These categories are well-represented in the training data.

## Known limitations

The training dataset is heavily imbalanced (e.g. ~30% buffer-overflow examples, mostly in memory-unsafe languages like C/C++, largely irrelevant to Python; some important categories like path traversal, hardcoded secrets, and weak cryptography have only a handful of examples total). As a result, in testing Cipheron **failed to correctly fix**:
- Path traversal
- Hardcoded secrets / API keys
- Weak hashing (e.g. MD5 for passwords)
- Insecure deserialization (`pickle.loads` on untrusted input)
- Reflected XSS

For these categories it tends to produce superficial, security-irrelevant changes (e.g. wrapping code in try/except, adding default arguments) rather than the actual fix. **Do not rely on this model as a substitute for a real security review or a larger model.** It's best used as a lightweight, offline first-pass check for the vulnerability classes listed above under "What it's good at," not as a general-purpose security auditor.

This is a small (0.5B parameter) educational/experimental model, not a production security tool.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("bencodez/Cipheron")
model = AutoModelForCausalLM.from_pretrained("bencodez/Cipheron", torch_dtype=torch.bfloat16)

messages = [
    {"role": "system", "content": "You are a secure coding assistant. Review code for security vulnerabilities and provide fixed, secure versions."},
    {"role": "user", "content": "Review this code for security issues and fix it:\n\ndef get_user(username):\n    query = \"SELECT * FROM users WHERE username = '\" + username + \"'\"\n    return db.execute(query)"},
]
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=False)
out = model.generate(input_ids, max_new_tokens=250)
print(tokenizer.decode(out[0][input_ids.shape[1]:], skip_special_tokens=True))
```

Or with the GGUF file via `llama-cpp-python` / llama.cpp / Ollama for lightweight CPU/on-device inference.

## License

Apache 2.0, inherited from the base model (Qwen2.5-Coder-0.5B-Instruct).