README.md

---
license: apache-2.0
tags:
- text-classification
- prompt-injection
- safety
- modernbert
datasets:
- deepset/prompt-injections
- Lakera/gandalf_ignore_instructions
- Lakera/mosscap_prompt_injection
- hackaprompt/hackaprompt-dataset
model-index:
- name: prompt-injection-lora
  results:
  - task:
      type: text-classification
      name: prompt-injection-detection
    dataset:
      name: jbb_behaviors
      type: ood-slate
    metrics:
    - type: auprc
      value: 0.5352
      name: AUPRC
      verified: false
  - task:
      type: text-classification
      name: prompt-injection-detection
    dataset:
      name: jbb_behaviors
      type: ood-slate
    metrics:
    - type: auroc
      value: 0.5284
      name: AUROC
      verified: false
  - task:
      type: text-classification
      name: prompt-injection-detection
    dataset:
      name: xstest
      type: ood-slate
    metrics:
    - type: auprc
      value: 0.4668
      name: AUPRC
      verified: false
  - task:
      type: text-classification
      name: prompt-injection-detection
    dataset:
      name: xstest
      type: ood-slate
    metrics:
    - type: auroc
      value: 0.53
      name: AUROC
      verified: false
  - task:
      type: text-classification
      name: prompt-injection-detection
    dataset:
      name: pooled_ood
      type: ood-slate
    metrics:
    - type: auprc
      value: 0.2934
      name: AUPRC
      verified: false
  - task:
      type: text-classification
      name: prompt-injection-detection
    dataset:
      name: pooled_ood
      type: ood-slate
    metrics:
    - type: auroc
      value: 0.383
      name: AUROC
      verified: false
---
# prompt-injection-lora — methodology submission rung

**Author**: Brandon Behring
**Date published**: 2026-05-18
**Project**: [https://github.com/brandon-behring/prompt-injection-detection-prototype](https://github.com/brandon-behring/prompt-injection-detection-prototype) at `v1.0.0`
**Submission audit ledger**: see `SUBMISSION_AUDIT.md` in the repo.
**Contamination tier (ADR-005 taxonomy)**: `backbone-partial-disjoint`.

This model card publishes the canonical fold0/seed42 checkpoint of the
`lora` rung from the methodology submission. The rung is one of a
5-rung ladder characterising what successive capability layers add to
prompt-injection detection across an IID test slate (4-source LODO
held-out positives) and a 5-slice OOD slate (BIPIA + InjecAgent +
JBB-Behaviors + XSTest + NotInject). **No rung is promoted as a
deployment recommendation** — each rung's trade-offs are characterised
per ADR-005 methodology-over-metrics framing.

## Intended use

Research-and-methodology-characterisation **only**. **NOT** production
deployment per ADR-005. The classifier-output behaviour is documented in
[the project WRITEUP](https://github.com/brandon-behring/prompt-injection-detection-prototype/blob/v1.0.0/WRITEUP.md) §5 + §7.

## Limitations

See
[the project's limitations spoke](https://github.com/brandon-behring/prompt-injection-detection-prototype/blob/v1.0.0/WRITEUP/limitations-and-future-work.md)
for the full list. Key points relevant to this checkpoint:

- LODO non-exchangeability (per assumption A-008) — train sets overlap
  across folds; per-fold variance reported in
  `evals/audit/cross_fold_ci_audit.parquet`.
- English-only; cross-language attacks out of scope per ADR-016.
- Single-class OOD slices (`bipia`, `injecagent`, `notinject`) have
  AUROC/AUPRC undefined per the project's WRITEUP §Methodology caveats
  convention; only `jbb_behaviors`, `xstest`, `pooled_ood` carry
  threshold-free ranking metrics.

## Headline results (canonical fold0/seed42; 95% BCa CI)

| Slice | AUPRC | AUROC |
|---|---|---|
| `jbb_behaviors` | 0.5352 [0.5042, 0.5633] | 0.5284 [0.5054, 0.5521] |
| `xstest` | 0.4668 [0.4465, 0.4857] | 0.5300 [0.5150, 0.5458] |
| `pooled_ood` | 0.2934 [0.2855, 0.3012] | 0.3830 [0.3737, 0.3925] |

Per-rung calibration (mean across folds × seeds):

| Slice | recall@FPR=1% (mean) | ECE (equal-mass) | Brier |
|---|---|---|---|
| `jbb_behaviors` | 0.0217 | 0.4721 | 0.4803 |
| `xstest` | 0.0150 | 0.4139 | 0.4245 |
| `pooled_ood` | 0.0000 | 0.4461 | 0.4484 |

Source: `evals/results.json` at v1.0.0 (BCa bootstrap per ADR-022,
10 000 resamples). Full per-rung × per-slice grid in the project
[WRITEUP §Results](https://github.com/brandon-behring/prompt-injection-detection-prototype/blob/v1.0.0/WRITEUP.md).

## Reproducibility (T0)

```bash
git clone https://github.com/brandon-behring/prompt-injection-detection-prototype
cd prompt-injection-detection-prototype
make install
make eval-from-hub RUNG=lora
```

This downloads the checkpoint, runs CPU eval against the local val slate,
and score-matches against `evals/results.json` within 1e-4 absolute per
ADR-034. ~10-30 min, $0 GPU.

Full T1 GPU re-eval via `make headline-cloud` (~$28 RunPod A100 80GB).

## Citation

```bibtex
@misc{behring2026promptinjectionlora,
  author       = {Behring, Brandon},
  title        = {prompt-injection-lora — methodology submission rung},
  year         = {2026},
  url          = { https://github.com/brandon-behring/prompt-injection-detection-prototype/tree/v1.0.0 }
}
```

## Linked ADRs

ADR-005 (contamination taxonomy), ADR-015 (single-backbone slate),
ADR-016 (data design), ADR-019 (transformer training recipe), ADR-032
(HF Hub publication discipline), ADR-034 (T0 reproducibility tier),
ADR-050 (rung-slate narrowing).