README.md

---
library_name: detectorscope
license: gemma
base_model: google/gemma-2-2b
pipeline_tag: text-classification
tags:
- ai-text-detection
- sparse-autoencoder
- interpretability
---

# detector-scope

Detects AI-generated text across domains and generators (trained on the COLING 2025 GenAI Detection shared task: news, essays, reviews and more, written by people or by 40 language models). The unmodified Gemma-2-2b, sparse autoencoders fine-tuned on this data, and a sparse logistic-regression head over their features: the best general-domain detector of the paper. Every prediction is a sum of contributions from sparse-autoencoder
features, each with a validated natural-language description, so the detector
can say which properties of the text drove its decision and where they occur.
Paper: [Building Auditable AI Text Detectors with Sparse Autoencoders](https://arxiv.org/abs/2610.30006);
code: [google-deepmind/detectorscope](https://github.com/google-deepmind/detectorscope).

## Use

```bash
pip install detectorscope
```

```python
from detectorscope import AuditableDetector

det = AuditableDetector.from_pretrained("liamdugan/detector-scope")
out = det("The authors propose a novel framework for ...")

out.p_ai       # probability that the text is AI-written
out.label      # "ai" or "human" (logit threshold +0.50)
out.features   # [Feature(id, importance, description, highlights), ...]
```

`out.features` lists every feature active in the text with a non-zero effect,
largest effect first. `importance` is how much the logit falls if that feature
is silenced: positive pushes toward AI, negative toward human, and
`out.logit == out.bias + sum(f.importance for f in out.features)`. A list of
texts returns a list of detections.

To see where a feature fires:

```python
out = det(text, return_highlights=True)
out.tokens                  # the text as a list of tokens
out.features[0].highlights  # [(token index, activation), ...] of the top feature
```

The default head is logistic regression over the features whose
auto-interpretation score is at least 0.85. `head="xgboost"` or another
`tau` (0.00 to 0.95 in steps of 0.05), on `from_pretrained` or on the call,
selects one of the other 39 heads.

## Performance

Default head; the decision threshold is fixed on the development data.

| split | AUROC | F1 | accuracy | documents |
|---|---|---|---|---|
| COLING dev | 0.932 | 0.867 | 0.839 | 261,758 |
| COLING dev-test | 0.916 | 0.866 | 0.854 | 32,557 |
| COLING test | 0.857 | 0.751 | 0.770 | 73,941 |

## What it looks at

The features with the largest average contribution on the training data
(weight × mean activation when active; density = share of the documents of
each class in which the feature fires).

**Toward AI**

| feature | description | mean contribution | AI density | human density |
|---|---|---|---|---|
| 13_3738 | introductory phrases or filler text commonly used by ai language models when declining personal questions | +0.987 | 25% | 2% |
| 13_1545 | academic and scientific research methodology and technical analysis terminology | +0.909 | 13% | 3% |
| 17_12178 | critical commentary or argumentative analysis of student policies and social responsibility | +0.558 | 38% | 11% |
| 17_8638 | neuron activates on random text fragments and noisy character sequences often found in garbled or fragmented data | +0.372 | 13% | 13% |
| 17_1175 | repeated phrases or sequences in text | +0.363 | 41% | 31% |
| 13_2153 | exclamation points used at the end of emotional or emphatic sentences | +0.318 | 16% | 15% |
| 22_9579 | markdown formatting for headers or list items | +0.297 | 25% | 13% |
| 7_1076 | conversational filler phrases used in informal speech | +0.261 | 15% | 13% |
| 7_7991 | reads more or similar web page navigation links | +0.239 | 8% | 7% |
| 17_4337 | introductory transition words signaling a summary or final thought | +0.210 | 25% | 7% |

**Toward human**

| feature | description | mean contribution | AI density | human density |
|---|---|---|---|---|
| 7_14967 | comma punctuation following a newline or preceding sentence start | -1.563 | 3% | 14% |
| 7_10954 | neuron fires on the digit 0 inside url placeholders like url 0 | -0.951 | 1% | 5% |
| 7_9535 | newline characters in scientific or technical text | -0.842 | 1% | 11% |
| 7_1262 | contractions containing the letter n and apostrophe specifically focusing on the n't ending | -0.785 | 3% | 14% |
| 7_5896 | newline characters in scientific or technical text | -0.658 | 1% | 11% |
| 13_11151 | newline characters in mathematical or technical contexts | -0.473 | 2% | 12% |
| 7_14666 | wikipedia category and navigation section headers | -0.454 | 4% | 10% |
| 22_3894 | semicolon followed by a newline character | -0.394 | 3% | 10% |
| 17_15548 | opening double quotation marks following a closing single quote and comma | -0.374 | 1% | 4% |
| 13_15646 | markdown editing and formatting indicators | -0.366 | 14% | 29% |

## Variants

Branches of this repo hold the paper's other two settings on the same data: `ft-model-sae` (Gemma-2-2b fine-tuned as a classifier, with SAEs fine-tuned on it) and `base-sae` (the unmodified Gemma-2-2b with the Gemma Scope SAEs). Load one with `revision="ft-model-sae"` or `revision="base-sae"`. For reviews of machine-learning papers use [detector-scope-peer-review-gpt4o](https://huggingface.co/liamdugan/detector-scope-peer-review-gpt4o) or [detector-scope-peer-review-llama](https://huggingface.co/liamdugan/detector-scope-peer-review-llama).

## Licence

The language model under `extractor/` is Gemma-2-2b, provided under
and subject to the [Gemma Terms of Use](https://ai.google.dev/gemma/terms)
(see `NOTICE`). The sparse autoencoders derive from
[Gemma Scope](https://huggingface.co/google/gemma-scope-2b-pt-res) (CC-BY-4.0);
the heads and feature descriptions are CC-BY-4.0.

## Citation

```bibtex
@misc{dugan2026detectorscope,
  title={Building Auditable AI Text Detectors with Sparse Autoencoders},
  author={Liam Dugan and Callum McDougall and Matthew Denton and Phillip Pham and Christine Kaeser-Chen and Chris Callison-Burch},
  year={2026},
  eprint={2610.30006},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2610.30006},
}
```