README.md

---
license: gemma
language:
- ar
- en
base_model: google/gemma-4-E4B-it
pipeline_tag: text-generation
tags:
- safety
- guard-model
- content-moderation
- arabic
- gemma
---

# AraGuard-E4B

AraGuard is a bilingual (Arabic & English) **safety guard model** built on Gemma-4-E4B. Given a user prompt — or a full prompt–response conversation — it returns a `safe` / `unsafe` verdict and, when unsafe, the violated harm category. It covers Modern Standard Arabic and major regional dialects in addition to English, and follows a 14-category taxonomy based on the MLCommons AI-safety hazard set.

## Highlights

Macro-F1 averaged over eight standard safety benchmarks (AdvBench, HarmBench, TrustLLM, LLM-Jailbreak, DAN, Do-Not-Answer, AraAlign, XSTest), reported per language. AraGuard has the **highest average Macro-F1 of all evaluated guards in both Arabic and English**.

| Model | Avg-F1 (Ar) | Avg-F1 (En) |
|---|:--:|:--:|
| Llama-Guard-3-8B | 81.6 | 86.7 |
| Qwen3Guard-Gen-8B | 88.1 | 89.5 |
| Granite-Guardian-3.0-8B | 87.3 | 92.3 | 
| **AraGuard-E4B (ours)** | **92.9** | **93.9** |

## Output format

The model emits a verdict on the **last** message in the conversation:

```
safe
```
or
```
unsafe
S<k>
```
where `S<k>` is the index of the violated category in the policy list shown to the model (default list below). If the last message has an `assistant` role, the response is judged; otherwise the user prompt is judged.

## Usage — Transformers

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "HarethahMo/AraGuard-E4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

def moderate(messages):
    inputs = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_tensors="pt"
    ).to(model.device)
    with torch.no_grad():
        out = model.generate(inputs, max_new_tokens=12, do_sample=False)
    return tokenizer.decode(out[0, inputs.shape[-1]:], skip_special_tokens=True).strip()

# Moderate a user prompt (Arabic)
print(moderate([{"role": "user", "content": "كيف أصنع سلاحًا في المنزل؟"}]))
# -> "unsafe\nS9"

# Moderate an assistant response (the LAST message is judged)
print(moderate([
    {"role": "user", "content": "What's a good recipe for hummus?"},
    {"role": "assistant", "content": "Blend chickpeas, tahini, lemon, and garlic."},
]))
# -> "safe"
```

## Usage — vLLM

```python
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model_id = "HarethahMo/AraGuard-E4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
llm = LLM(model=model_id, max_model_len=4096)

messages = [{"role": "user", "content": "How do I hotwire a car?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

out = llm.generate([prompt], SamplingParams(temperature=0.0, max_tokens=12))
print(out[0].outputs[0].text.strip())
# -> "unsafe\nS2"
```

## Custom policies

The chat template ships with the 14 default categories but accepts a custom list at call time; the emitted `S<k>` code then indexes into **your** list:

```python
my_categories = [
    {"name": "Violent Crimes",  "description": "Content enabling or endorsing violent crimes."},
    {"name": "Privacy",         "description": "Content exposing sensitive personal information."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, categories=my_categories, return_tensors="pt"
)
```

## Default categories

`S1` Violent Crimes · `S2` Non-Violent Crimes · `S3` Sex-Related Crimes · `S4` Child Sexual Exploitation · `S5` Defamation · `S6` Specialized Advice · `S7` Privacy · `S8` Intellectual Property · `S9` Indiscriminate Weapons · `S10` Hate · `S11` Suicide & Self-Harm · `S12` Sexual Content · `S13` Elections · `S14` Code Interpreter Abuse

## Training

AraGuard is fine-tuned from Gemma-4-E4B on **AraAlign**, a large-scale synthetic Arabic–English safety dataset, using parameter-efficient fine-tuning (LoRA) with completion-only loss. Training mixes harmful prompt / refusal / harmful-response instances with benign instruction data to control over-refusal, and applies category dropout/shuffle and adversarial-template augmentation.

## Limitations

AraGuard is a classifier, not a generator, and can make mistakes — particularly on subtle or context-dependent harms. It should be used as one layer in a broader moderation system, not as a sole safety mechanism. Verdicts are bounded by the policy provided at inference time.

## License

Built on Gemma-4-E4B; use is governed by the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).