---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-0.8B
pipeline_tag: text-classification
library_name: transformers
language:
- en
- pl
tags:
- nvfp4
- fp4
- compressed-tensors
- vllm
- quantized
- tentaguard
- guard
- security
- prompt-injection
- tentaflow
---

# TentaGuard — NVFP4 (W4A4, vLLM)

**TentaGuard** is a lightweight security classifier (guard) — a fine-tune of
[`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B). It is used **mainly inside the
[TentaFlow](https://github.com/Slyb00ts/TentaFlow) application** to scan external content — messages, documents,
web-search results, etc. — for **hidden attacks** (prompt injection / jailbreak) before it
reaches the main LLM.

The model does NOT generate user-facing replies — it returns a single digit:

| Label | Meaning |
|-------|---------|
| `0` | benign (safe content) |
| `1` | prompt injection / tool abuse (technical attack) |
| `2` | jailbreak (behavioural manipulation) |

If the text contains BOTH injection and jailbreak → `1`.

## Input format

A classifier system prompt + a user message `<|guard|>\n{text}`. **Build the prompt with the
model tokenizer (`apply_chat_template`)** — do not rely on a generic chat template.

## Accuracy (guard test set)

- Exact (0/1/2): **~96.6%** (full precision) / **~94.8%** (Q5_K_M)
- Safe / Unsafe: **~98.3%**

## Authors

Trained by: **Katarzyna Nowak**, **Piotr Jarocki**, **Damian Pala**, **Jakub Rurański**.

## License & attribution

Apache-2.0, inherited from the base model [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B).
This checkpoint is a fine-tune for attack detection, built for the [TentaFlow](https://github.com/Slyb00ts/TentaFlow) application.

## Usage (vLLM)

`compressed-tensors` format (`nvfp4-pack-quantized`): 4-bit weights (FP4 E2M1, groups of 16,
FP8 E4M3 block scales + a global FP32 scale), 4-bit activations (W4A4), `lm_head` kept in full
precision. PTQ calibration via [`llm-compressor`](https://github.com/vllm-project/llm-compressor)
on real guard prompts.

NVFP4 is hardware-accelerated on **Blackwell (sm_100+)**; on older GPUs vLLM loads it as
**weight-only** (smaller VRAM, no FP4 acceleration).

```bash
vllm serve TentaFlow/TentaGuard-NVFP4
```