README.md

---
language:
  - he
  - en
license: apache-2.0
library_name: transformers
base_model: BrainboxAI/cyber-analyst-4B
base_model_relation: finetune
pipeline_tag: text-generation
tags:
  - cybersecurity
  - security
  - vulnerability-verification
  - finding-verifier
  - bug-bounty
  - triage
  - hebrew
  - israel
  - gemma4
  - unsloth
  - lora
  - gguf
  - pilot
pretty_name: Cyber-Analyst 4B (Verifier Pilot)
---

# Cyber-Analyst 4B: Verifier Pilot

A LoRA fine-tune of [`cyber-analyst-4B`](https://huggingface.co/BrainboxAI/cyber-analyst-4B) that adds a strict finding-verifier behavior. Given the evidence for a suspected web vulnerability, it decides whether the evidence actually proves the bug, and rejects the ones that only look real.

[![Base](https://img.shields.io/badge/Base-cyber--analyst--4B-yellow)](https://huggingface.co/BrainboxAI/cyber-analyst-4B)
[![Method](https://img.shields.io/badge/Method-bf16_LoRA_(Unsloth)-orange)](https://github.com/unslothai/unsloth)
[![Format](https://img.shields.io/badge/Format-GGUF_·_LoRA_·_safetensors-blue)]()
[![Stage](https://img.shields.io/badge/Stage-PILOT-red)]()
[![License](https://img.shields.io/badge/License-Apache_2.0-lightgrey)](https://www.apache.org/licenses/LICENSE-2.0)

> [!WARNING]
> This is a pilot. It was trained on a small (~100 example) balanced set to check the
> recipe before scaling. It lives in its own repo and does not replace the production
> [`cyber-analyst-4B`](https://huggingface.co/BrainboxAI/cyber-analyst-4B).

## What it does

You give it the evidence for a suspected vulnerability (tool, URL, payload, response
snippet, PoC). It returns a JSON verdict:

```json
{"verdict": "true_positive | false_positive | inconclusive", "reason": "..."}
```

It judges the evidence, not the scary-looking payload. That is the difference between
a verifier and a scanner. It is trained to reject the look-alikes that fool most tools:

| Looks like a finding | Why it is a false positive |
|---|---|
| SQLi, got a 500 | a generic 500 with no DB error and no extracted data proves nothing |
| blind boolean SQLi | the true and false conditions returned identical responses |
| reflected XSS | the payload came back HTML-encoded or JS-escaped, so it cannot run |
| SSRF to a metadata IP | `ECONNREFUSED`. the request was blocked, no internal data returned |
| IDOR by enumerating IDs | every non-owned ID returned 403; authorization is enforced |
| path traversal with a null byte | 404, no `/etc/passwd` contents; the bypass read no sensitive file |
| JWT with alg=none | the server rejected it with 401 |

When something is genuinely suspicious but not confirmed, it answers `inconclusive` and
says what evidence is still missing (a differential test, a decoded token, an extracted
row). It also knows who it is (Cyber-Analyst 4B, by BrainboxAI), in Hebrew and English.

## Lineage

```
Google Gemma-4 E4B
  -> cyber-analyst-4B          (1.27M security examples, BrainboxAI)
       -> this pilot           (verifier + identity LoRA delta)
```

## Training

| | |
|---|---|
| Base | `BrainboxAI/cyber-analyst-4B` (weights kept; this only adds a delta) |
| Method | bf16 LoRA (Unsloth), r=16 / alpha=32, LR 1e-4, 3 epochs, cosine, seq 2048 |
| Data | ~100 balanced verifier examples (true / false / inconclusive, with adversarial hard negatives across 19 vuln classes) plus a small share of bilingual identity |
| Eval | held-out split, `eval_loss` around 1.13 |
| Hardware | one RTX 4090, ~16GB VRAM |

## Benchmark: does the delta actually reject false positives?

The whole point of the verifier delta is to stop the model from confirming
vulnerabilities that only look real. So the benchmark measures exactly that.

17 held-out examples (never seen in training): 8 true positives, 6 false
positives, 3 inconclusive. Each is a suspected finding with its evidence. The
model returns a verdict. Thinking mode is disabled with an assistant prefill so
the model answers directly. Same test, same prompt, run against the base and the
pilot.

![Base vs pilot on 17 held-out examples](benchmark.png)

The base model confirms everything. It says `true_positive` to all 8 real bugs,
but also to all 6 look-alikes: a 500 with no DB error, an SSRF that got
`ECONNREFUSED`, an IDOR where every non-owned ID returned 403. It cannot tell a
finding from a decoy. That is the scanner bias a verifier is supposed to remove.

After 100 verifier examples the pilot rejects 4 of those 6 false positives, and
overall accuracy goes from 47% to 65%. The cost was one over-correction: it
flagged a genuine Twig SSTI as a false positive. A fair trade for a delta this
small, and the direction is the one we wanted.

What it still gets wrong: neither model abstains. Both answer 0 of 3 inconclusive
cases, because the pilot inherited the base model's habit of always committing to
a verdict, and 100 examples was not enough to install "say inconclusive when the
confirming evidence is missing." That is the first thing the next round fixes,
with more inconclusive training data.

Read this as what it is: a pilot proving the recipe works. A 100-example delta
moved false-positive rejection from 0/6 to 4/6 without breaking true-positive
detection. Scale the data and the abstention gap should close too.

## Files

| File | What | Use |
|---|---|---|
| `cyber-analyst-4B-verifier-pilot.Q4_K_M.gguf` | quantized model, ~5.3 GB | LM Studio / Ollama / llama.cpp |
| `cyber-analyst-4B-verifier-pilot.mmproj-BF16.gguf` | vision projector | only for image input (Gemma-4 is multimodal) |

The LoRA adapter lives in a separate repo, [`cyber-analyst-4B-verifier-pilot-lora`](https://huggingface.co/BrainboxAI/cyber-analyst-4B-verifier-pilot-lora), so this one stays a clean GGUF repo.

Note: Gemma-4 is a new architecture. Loading the GGUF needs a recent llama.cpp, so
update LM Studio or Ollama to the latest. In LM Studio, make sure the format selector
is set to GGUF.

## Quick start (LM Studio)

Download `gguf/cyber-analyst-4B-safetensors.Q4_K_M.gguf` and load it. For the verifier
task, use a system prompt like:

> You are a strict security-finding verifier. Given the evidence for a suspected
> vulnerability, decide `true_positive` (the evidence proves exploitation),
> `false_positive` (disproven or not exploitable), or `inconclusive` (suspicious but
> the confirming evidence is missing; say what is missing). Judge from the evidence,
> not the title. Respond with only JSON: `{"verdict":"…","reason":"…"}`.

## Continue training from here

Load the base `cyber-analyst-4B` and attach the adapter from
[`cyber-analyst-4B-verifier-pilot-lora`](https://huggingface.co/BrainboxAI/cyber-analyst-4B-verifier-pilot-lora)
to keep training. The cleaner option is to train from the original `cyber-analyst-4B`
with the full accumulated dataset, which avoids stacking one delta on top of another.
A merged 16-bit model can be regenerated from the base plus the adapter in one step
when you need it.

## Limitations

- Pilot scale, about 100 examples. It checks the format and basic discrimination. It is
  not a production verifier yet. Scale the dataset (600+) before relying on it.
- It inherits the base model's limits (public-data bias, temporal cutoff).
- Not for autonomous security decisions without human review.

## Citation

```bibtex
@misc{elyasi2026cyberverifierpilot,
  title  = {Cyber-Analyst 4B: Verifier Pilot},
  author = {Elyasi, Netanel},
  year   = {2026},
  publisher = {BrainboxAI},
  howpublished = {\url{https://huggingface.co/BrainboxAI/cyber-analyst-4B-verifier-pilot}}
}
```

## Author

Built by [Netanel Elyasi](https://huggingface.co/BrainboxAI), founder of [BrainboxAI](https://brainboxai.io). Contact: netanele@brainboxai.io