Instructions to use Guy-Amit/gemma4-e2b-safety-decision-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Guy-Amit/gemma4-e2b-safety-decision-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("google/gemma-4-E2B-it") model = PeftModel.from_pretrained(base_model, "Guy-Amit/gemma4-e2b-safety-decision-lora") - Notebooks
- Google Colab
- Kaggle
Gemma 4 E2B decision model β LoRA
This repository contains the LoRA adapter trained on the 500k decision dataset.
It is designed for the SystemOne structured-decision API rather than ordinary
free-form generation. The merged version is available at Guy-Amit/gemma4-e2b-safety-decision.
Load with PEFT
from peft import PeftModel
from transformers import Gemma4ForConditionalGeneration
base = Gemma4ForConditionalGeneration.from_pretrained(
"google/gemma-4-E2B-it", dtype="auto"
)
model = PeftModel.from_pretrained(base, "Guy-Amit/gemma4-e2b-safety-decision-lora")
For vLLM, serve google/gemma-4-E2B-it with --enable-lora and point
--lora-modules decision=Guy-Amit/gemma4-e2b-safety-decision-lora at this adapter.
Evaluation results
Held-out test-set results computed through the vLLM SystemOne endpoint. Accuracy is top-1 accuracy over legal decision labels. Target mass at prediction gives partial credit when an example has a probability distribution over targets.
Overall
| Split | Examples | Loss | Accuracy | Target mass at prediction |
|---|---|---|---|---|
| Overall | 50,181 | 0.6169 | 75.43% | 75.45% |
By decision format
| Split | Examples | Loss | Accuracy | Target mass at prediction |
|---|---|---|---|---|
| noul | 14,132 | 0.2086 | 92.24% | 92.05% |
| choice | 11,989 | 0.4976 | 82.48% | 82.80% |
| score | 24,060 | 0.9163 | 62.04% | 62.03% |
By data source
| Split | Examples | Loss | Accuracy | Target mass at prediction |
|---|---|---|---|---|
| aegis | 688 | 0.3558 | 85.03% | 85.03% |
| agent_failure_bench | 229 | 0.0874 | 96.51% | 96.51% |
| ai_agent_security_policy_decisions | 240 | 0.2622 | 91.67% | 91.67% |
| arc | 200 | 0.4571 | 85.50% | 85.50% |
| banking77 | 500 | 2.1037 | 14.00% | 14.00% |
| boolq | 229 | 0.3431 | 86.90% | 86.90% |
| civil_comments | 3,206 | 0.0869 | 98.16% | 97.31% |
| clinc150 | 500 | 0.0503 | 98.00% | 98.00% |
| commonsense_qa | 500 | 0.8079 | 72.20% | 72.20% |
| dynabench | 219 | 0.7022 | 65.75% | 65.75% |
| helpsteer2 | 2,500 | 0.9837 | 59.28% | 59.28% |
| multi_nli | 500 | 0.4630 | 83.00% | 83.00% |
| natural_questions | 229 | 0.5742 | 74.67% | 74.67% |
| nemotron | 5,200 | 0.2516 | 90.65% | 90.65% |
| oasst1 | 499 | 0.0720 | 98.40% | 98.40% |
| open_jev | 6,909 | 0.2732 | 88.25% | 88.75% |
| policy_adherence_4k | 757 | 0.0861 | 96.83% | 96.83% |
| prometheus | 4,000 | 0.7306 | 69.45% | 69.45% |
| qasc | 500 | 0.0609 | 98.20% | 98.20% |
| snli | 500 | 0.3876 | 86.80% | 86.80% |
| squad_v2 | 229 | 0.4024 | 83.84% | 83.84% |
| stepguard | 393 | 1.2583 | 64.89% | 64.89% |
| toolace | 853 | 0.0204 | 99.30% | 99.30% |
| toolcall_guard | 412 | 0.9736 | 70.63% | 70.63% |
| tydiqa | 229 | 0.3878 | 86.90% | 86.90% |
| ultrafeedback | 15,976 | 1.0269 | 57.45% | 57.45% |
| wildguard | 2,985 | 0.5926 | 83.02% | 83.02% |
| xlam | 999 | 0.0125 | 99.90% | 99.90% |
Evaluation covered 50,181 examples at 48.96 examples/second.
- Downloads last month
- -