teptez-ai

SAST finding triage model — fine-tuned on real production security scan data to classify Static Application Security Testing findings as true positives, false positives, or uncertain, with CWE labels, confidence scores, and remediation guidance.

---

Overview

Rule-based SAST tools generate enormous volumes of findings, a significant portion of which are false positives. Security analysts spend hours triaging noise instead of fixing real vulnerabilities. teptez-ai is a 7B-parameter LLM fine-tuned specifically to automate this triage step.

Given a SAST finding (title, CWE, severity, code snippet, taint flow), teptez-ai returns a structured JSON verdict:

• true_positive — finding is real, exploit path exists
• false_positive — finding is noise, safe to suppress
• uncertain — insufficient context, escalate to analyst

Fine-tuned on production findings from the Teptez security platform — real codebases, real scan data, real analyst labels.

Key specs

Property	Value
Base model	Qwen2.5-Coder-7B-Instruct
Quantization	GGUF Q4_K_M
Model size	~4.7 GB
Inference speed	~100 tok/s (RTX 3090 24GB)
Context window	8192 tokens
License	Apache 2.0

---

Benchmark Results

Evaluated against OWASP Benchmark v1.2 — the standard industry benchmark for SAST tools — using the official Youden's J statistic (J = TPR − FPR).

J = 0.0 is random. J = 1.0 is perfect. Open-source SAST tools typically score 0.30–0.45.

Head-to-head vs base model

Model	TPR	FPR	Youden J	vs base
teptez-ai (Q4_K_M)	0.68	0.57	0.109	+0.048 (+79%)
qwen2.5-coder-7b (base)	0.66	0.60	0.061	—

Fine-tuning delivers a 79% relative improvement in Youden's J over the base model, primarily by cutting the false positive rate from 0.60 to 0.57 across the full benchmark.

Per-category breakdown

CWE Category	teptez-ai J	base J	Delta
Command Injection (CWE-78)	0.40	0.22	+0.18
SQL Injection (CWE-89)	0.33	0.18	+0.15
XSS (CWE-79)	0.28	0.12	+0.16
Path Traversal (CWE-22)	0.22	0.09	+0.13
Weak Randomness (CWE-330)	0.13	0.05	+0.08
Crypto/Hash (CWE-327/328)	0.00	0.01	-0.01
Auth/Authz (CWE-862/639)	0.02	0.01	+0.01
Timing (CWE-208)	0.05	0.04	+0.01
Secure Cookie (CWE-614)	-0.08	0.52	-0.60 ⚠️

Strong on injection classes. teptez-ai significantly outperforms the base model across all injection-type CWEs (cmdi/sqli/xss/path/weakrand). These are the highest-volume SAST categories in real codebases.

Securecookie regression. CWE-614 (missing HttpOnly/Secure flags) shows a severe regression vs the base model. Do not use teptez-ai to triage cookie security findings. This is a known training artifact being fixed in the next round.

Dead categories. Crypto, authz, and timing categories have near-zero Youden J on both models — 7B parameters are insufficient for these without full class context. Escalate to frontier models or human analysts.

Production run

On 369 real production SAST findings from live codebases:

• 17% rejected as false positive (~63 findings suppressed)
• Injection-class findings: majority of suppressions, generally accurate
• Authz/crypto findings: some wrong suppressions (do not enable for these categories)

---

Usage

Recommended architecture

Use teptez-ai as a gated FP suppressor, not a confirmer:

Rule-engine finding
        │
        ▼
Is CWE in injection classes?  ──No──▶  Keep finding (don't run model)
        │ Yes
        ▼
Run teptez-ai with full function + taint context
        │
        ├── verdict: false_positive, confidence > 0.75  ──▶  Suppress finding
        ├── verdict: true_positive                       ──▶  Keep finding
        └── verdict: uncertain / confidence < 0.75      ──▶  Escalate to frontier model / analyst

Only suppress on false_positive — never on true_positive. The model is biased toward flagging (FPR 0.57), so a false_positive verdict is rare and higher-precision.

Injection-class CWEs only (where Youden J ≥ 0.13):

• CWE-78 Command Injection
• CWE-79 Cross-Site Scripting
• CWE-89 SQL Injection
• CWE-22 Path Traversal
• CWE-330 Weak Randomness

Never auto-suppress:

• CWE-614 Secure Cookie (regression — model worse than random)
• CWE-327/328 Weak Crypto/Hash (near-zero J)
• CWE-862/639 Auth/Authz/IDOR (near-zero J)
• CWE-208 Timing Attacks (near-zero J)

Running with llama.cpp / Ollama

# Pull via Ollama
ollama pull hf.co/Kuyash/teptez-ai:Q4_K_M

# Or run directly with llama.cpp
./llama-cli -m teptez-ai-Q4_K_M.gguf \
  --temp 0.1 \
  --top-p 0.9 \
  -n 512 \
  -p "<prompt>"

Python integration

import json
import requests

def triage_finding(finding: dict) -> dict:
    prompt = f"""<|im_start|>system
You are a SAST triage expert. Analyze this finding and return JSON with keys:
verdict (true_positive|false_positive|uncertain), confidence (0.0-1.0),
cwe (string), explanation (string), remediation (string).
<|im_end|>
<|im_start|>user
Finding: {finding['title']}
CWE: {finding.get('cwe_id', 'unknown')}
Severity: {finding.get('severity', 'MEDIUM')}
Code:
{finding.get('code_snippet', '')}

Taint flow: {finding.get('data_flow', 'not available')}
<|im_end|>
<|im_start|>assistant
"""
    response = requests.post("http://localhost:11434/api/generate", json={
        "model": "teptez-ai",
        "prompt": prompt,
        "stream": False,
        "options": {"temperature": 0.1}
    })
    text = response.json()["response"].strip()
    # Strip markdown fences if present
    if text.startswith("```"):
        text = text.split("```")[1]
        if text.startswith("json"):
            text = text[4:]
    return json.loads(text)

# Gate: only run on injection CWEs
INJECTION_CWES = {"CWE-78", "CWE-79", "CWE-89", "CWE-22", "CWE-330"}

def should_suppress(finding: dict) -> bool:
    cwe = finding.get("cwe_id", "")
    if cwe not in INJECTION_CWES:
        return False  # don't touch non-injection
    result = triage_finding(finding)
    return (
        result.get("verdict") == "false_positive"
        and result.get("confidence", 0) >= 0.75
    )

---

Input / Output Format

Prompt template

<|im_start|>system
You are a SAST triage expert. Analyze this finding and return JSON.
<|im_end|>
<|im_start|>user
Finding: {title}
CWE: {cwe_id}
Severity: {severity}
Code:
{code_snippet}

Taint flow: {data_flow}
<|im_end|>
<|im_start|>assistant

Tips for best results:

• Provide the full function, not just the flagged line — avoids "insufficient context" errors
• Include taint flow when available (source → sink path from your SAST tool)
• Keep code under 2048 tokens; truncate from the bottom if needed

Output schema

{
  "verdict": "true_positive" | "false_positive" | "uncertain",
  "confidence": 0.85,
  "cwe": "CWE-89",
  "explanation": "User input from request.getParameter() flows directly into a string-concatenated SQL query with no parameterization or escaping.",
  "remediation": "Replace string concatenation with a PreparedStatement: `conn.prepareStatement(\"SELECT * FROM users WHERE id = ?\")` and bind the parameter with `stmt.setString(1, userId)`."
}

Field	Type	Description
verdict	string	true_positive, false_positive, or uncertain
confidence	float	0.0–1.0; scores < 0.75 should be treated as uncertain
cwe	string	Classified CWE identifier
explanation	string	Why the model reached this verdict
remediation	string	Concrete fix recommendation

---

Limitations

Known issues (as of current release)

CWE-614 Secure Cookie — severe regression. teptez-ai scores Youden J = −0.08 on secure cookie findings, compared to 0.52 for the base model. This is a catastrophic regression caused by training data imbalance. Do not use teptez-ai for HttpOnly/Secure flag findings until this is fixed.

High overall FPR (0.57). The model over-flags — it sees vulnerability in safe code, especially in crypto, auth, and cookie-related code patterns. A false_positive verdict is more reliable than a true_positive verdict because it swims against the model's bias.

Dead categories (crypto/authz/timing). CWE-327/328/614/862/639/208 have near-zero Youden J. The model lacks sufficient training signal for these categories. Use a frontier model (Claude, GPT-4o, Gemini) or a human analyst for these.

7B parameter ceiling. Subtle IDOR, broken access control, and privilege escalation patterns require understanding class hierarchy, authentication flow, and business logic across multiple files. A 7B model with single-function context cannot reliably detect these.

GGUF Q4_K_M quantization. ~4-bit quantization introduces slight accuracy loss vs fp16. For maximum accuracy, use the Q8_0 variant (if available) or the full fp16 model.

Snippet-only inputs fail. If you pass only the flagged 1–3 lines without the surrounding function, the model frequently returns uncertain with "insufficient context." Always include the full function body.

---

Reproducing the Benchmark

# 1. Clone OWASP Benchmark
git clone https://github.com/OWASP-Benchmark/BenchmarkJava
cd BenchmarkJava && mvn package -DskipTests

# 2. Run evaluation (requires teptez-ai running on Ollama)
python salad/eval/owasp_eval.py      # stratified sample → results.jsonl
python salad/eval/owasp_score.py     # Youden J + per-category table
python salad/eval/owasp_compare.py   # head-to-head vs base model

The eval script uses a stratified sample of OWASP BenchmarkJava test cases, covering all CWE categories proportionally. Scoring follows the official OWASP methodology (Youden's J = TPR − FPR).

---

Roadmap

The following improvements are planned for the next fine-tuning round:

Round 2 targets

Improvement	Target metric
Flood training with safe-code negatives (50/50 balance)	FPR < 0.30
Fix securecookie regression (restore CWE-614 training data)	J(CWE-614) > 0.40
Add dead-category examples (crypto/authz/timing)	J(CWE-327/328) > 0.10
Calibrate confidence score (train explicit uncertain label)	Confidence Brier score < 0.15
Full-function context at train AND inference	Reduce "uncertain" on short snippets

Root cause of FPR problem: Current training set is vuln-heavy (more vulnerable examples than safe ones). The model learned to flag aggressively. Rebalancing to 50/50 with explicit safe variants (parameterized SQL, escaped HTML, validated paths, compare_digest timing-safe comparisons, role-checked endpoints, strong ciphers) is the single highest-leverage fix.

Securecookie fix: Restore the original training examples for CWE-614 that were accidentally dropped. Mix with new negative examples. Lower learning rate for this category to avoid forgetting again.

No catastrophic forgetting: Round 2 will mix old injection data with new negatives and use a lower learning rate on the balanced set, following standard continual learning practice.

Round 3 vision

• Full-function + cross-file taint context (requires longer context fine-tune)
• Multi-label output (multiple CWEs per finding)
• Confidence calibration verified against held-out production labels
• Youden J > 0.25 across all injection categories
• FPR < 0.30 overall

---

Citation

If you use teptez-ai in your research or tooling, please cite:

@misc{teptez-ai-2026,
  title     = {teptez-ai: A Fine-Tuned LLM for SAST Finding Triage},
  author    = {Teptez Security},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Kuyash/teptez-ai}
}

Evaluated against OWASP Benchmark v1.2 using the official Youden's J scoring methodology.

---

Related

• OWASP Benchmark — the benchmark used for evaluation
• Qwen2.5-Coder-7B-Instruct — the base model

---

teptez-ai is a security research model. Results may vary across codebases and languages. Always have a human analyst review suppressed findings in critical security contexts.