---
base_model: google/gemma-4-E4B-it
datasets:
- liamdugan/raid
library_name: transformers
tags:
- ai-text-detection
- detection-tokens
- neologism
- raid
---

# Gemma-4-E4B RAID-only CE detection tokens

This repository contains the locked RAID-only pairwise-CE detection-token
checkpoints for `google/gemma-4-E4B-it`. The detector score for a text `x` is

```text
mean_logp(x | "Write <ai> text.") - mean_logp(x | "Write <human> text.")
```

The default files `ai_token.pt` and `human_token.pt` are the seed-707
checkpoint, selected because it has the highest full-BEEMO AUROC among the
three completed seeds. Complete checkpoints for seeds 42, 101, and 707 are in
`seeds/`.

## Locked training procedure

- Data: RAID `standard_train_expanded6` only.
- Held-out evaluation: RAID `standard_test`; train/test source-ID overlap is
  asserted to be zero.
- Objective: reference-free pairwise logistic CE,
  `softplus(-(mean_logp(ai_text | AI prompt) -
  mean_logp(human_text | AI prompt)))`.
- Trainable parameters: the main `<ai>` embedding row only. The `<human>` row,
  all backbone weights, and Gemma's auxiliary embedding rows remain fixed.
- Prompt: `Write {token} text.`
- Sequence score: average continuation-token log likelihood.
- Maximum length: 512 tokens.
- Batch size: 8.
- Pilot: 500 randomly sampled RAID pairs, 2 epochs (125 updates), AdamW,
  peak LR `1e-3`, 20-update linear warmup, cosine decay to `1e-5`.
- Continuation: 5,000 randomly sampled RAID pairs, 75 updates, peak LR
  `1e-4`, 20-update warmup, using a 625-update cosine schedule horizon and
  stopping after update 75.
- Seeds: 42, 101, and 707; the seed controls RAID sampling and evaluation
  bootstrap resampling.
- Evaluation: all 2,163 eligible BEEMO pairs and all 3,000 RAID standard-test
  pairs, with 1,000 paired bootstrap resamples per seed.

The canonical model-aware entry point is
`scripts/run_raid_ce_canonical.py`. It also contains the locked Llama profile;
the lower Llama learning rate and omission of continuation are explicit in
that profile.

## Results

| Seed | BEEMO AUROC | RAID test AUROC |
|---:|---:|---:|
| 42 | 0.7549 | 0.9642 |
| 101 | 0.7565 | 0.9503 |
| 707 | **0.7689** | 0.9599 |
| Mean +/- sample SD | **0.7601 +/- 0.0077** | **0.9581 +/- 0.0071** |

Per-seed paired-bootstrap confidence intervals and complete metric fields are
stored in `metrics.json`.

## Files

```text
ai_token.pt
human_token.pt
seeds/{42,101,707}/{ai_token.pt,human_token.pt,summary.json}
metrics.json
training_config.json
scripts/run_raid_ce_canonical.py
scripts/run_gemma_expanded_pairwise_ce.py
detection_tokens/
```

Each token checkpoint includes the main embedding and Gemma auxiliary
embedding row needed by the loader. These are embedding-only artifacts and
require access to the Gemma backbone.