Text Classification
PEFT
Safetensors
guardrails
content-safety
prompt-injection
jailbreak-detection
legal-advice
tax-advice
decision-model
embeddings
lora
gemma
Instructions to use alinia/Sev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use alinia/Sev with PEFT:
from peft import PeftModel from transformers import AutoModel base_model = AutoModel.from_pretrained("google/gemma-4-E4B-it") model = PeftModel.from_pretrained(base_model, "alinia/Sev") - Notebooks
- Google Colab
- Kaggle
Card: batch drift up to 0.09 as measured; legal-label note without private-set numbers
Browse files
README.md
CHANGED
|
@@ -174,8 +174,8 @@ and false-positive rate at 0.5 are the untuned default.
|
|
| 174 |
PolyGuardMix (which contains WildGuardMix, the source of PolyGuard's prompts) and deepset. Near-copies
|
| 175 |
of training prompts were removed from the test sets, but these four sets are in-distribution for Sev. Tev1-4B and Kev-4B were trained on general decision data, not on safety or
|
| 176 |
security data, and Jev's training data is not disclosed.
|
| 177 |
-
- **Legal advice is hard to label.**
|
| 178 |
-
|
| 179 |
|
| 180 |
## Choosing a threshold
|
| 181 |
|
|
@@ -276,7 +276,7 @@ the same rows.
|
|
| 276 |
- **Thresholds.** See above: tax and legal need a higher threshold than 0.5.
|
| 277 |
- **Long inputs.** The text is cut to its first 1,024 tokens. Training texts are single messages or one
|
| 278 |
user message with its reply; long multi-turn transcripts are out of distribution.
|
| 279 |
-
- **Batching.** In bf16, the same text can score up to about 0.
|
| 280 |
near the middle of the range, from padding numerics; scores near 0 or 1 do not move.
|
| 281 |
- **Financial and medical advice** were trained on synthetic data only and have not been evaluated on
|
| 282 |
human-labelled data.
|
|
|
|
| 174 |
PolyGuardMix (which contains WildGuardMix, the source of PolyGuard's prompts) and deepset. Near-copies
|
| 175 |
of training prompts were removed from the test sets, but these four sets are in-distribution for Sev. Tev1-4B and Kev-4B were trained on general decision data, not on safety or
|
| 176 |
security data, and Jev's training data is not disclosed.
|
| 177 |
+
- **Legal advice is hard to label.** Whether a reply gives advice to this person or explains the law in
|
| 178 |
+
general is where human reviewers disagree most, so small gaps on legal are noise.
|
| 179 |
|
| 180 |
## Choosing a threshold
|
| 181 |
|
|
|
|
| 276 |
- **Thresholds.** See above: tax and legal need a higher threshold than 0.5.
|
| 277 |
- **Long inputs.** The text is cut to its first 1,024 tokens. Training texts are single messages or one
|
| 278 |
user message with its reply; long multi-turn transcripts are out of distribution.
|
| 279 |
+
- **Batching.** In bf16, the same text can score up to about 0.09 differently at batch 1 and batch 16
|
| 280 |
near the middle of the range, from padding numerics; scores near 0 or 1 do not move.
|
| 281 |
- **Financial and medical advice** were trained on synthetic data only and have not been evaluated on
|
| 282 |
human-labelled data.
|