FlorianJK/Meta-Llama-3.1-8B-SecAlign-pp

Meta-Llama-3.1-8B-Instruct — SecAlign++ Defend Adapter

A PEFT LoRA adapter for meta-llama/Llama-3.1-8B-Instruct fine-tuned with SecAlign++ to make the model resistant to prompt injection attacks.

Model Details

• Base model: meta-llama/Llama-3.1-8B-Instruct
• Fine-tuning method: DPO (Direct Preference Optimisation) via SecAlign++
• Adapter type: PEFT LoRA
• LoRA rank / alpha: 32 / 8
• LoRA target modules: q_proj, v_proj, gate_proj, up_proj, down_proj
• Training data: 19,157 samples from the Alpaca dataset (synthetic_alpaca) with self-generated model responses and randomly-injected adversarial instructions
• Epochs: 3 · Batch size: 1 · Gradient accumulation steps: 16 · LR: 1.6 × 10⁻⁴
• dtype: bfloat16

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "FlorianJK/Meta-Llama-3.1-8B-SecAlign-pp")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")

Method

SecAlign++ extends SecAlign with:

1. Self-generated responses — the model's own outputs are used as the rejected/chosen pair, rather than fixed templates, making the preference signal more model-specific.
2. Randomised injection position — the adversarial instruction is inserted at a random position within the data section during training, increasing robustness across injection locations.

The adapter teaches the model to produce its normal helpful answer (chosen) and ignore any injected instruction (rejected) via DPO with β = 0.1.

Related Models

Model	Description
FlorianJK/Meta-Llama-3.1-8B-SecUnalign-pp	Same architecture fine-tuned with inverted preferences — intentionally vulnerable to prompt injection (attack / red-team adapter)
FlorianJK/Meta-Llama-3-8B-SecAlign	SecAlign adapter for the older Llama 3 8B base