README.md

---
license: mit
language:
- en
tags:
- text
- conversations
- classification
pretty_name: Slop classifier dataset
---

# Slop classifier dataset

A human-annotated dataset for studying and classifying **AI-generated text that people perceive as “AI slop.”**

The dataset is built from samples collected from existing public datasets and annotated through the **Bench Labs SlopFinder** interface.

## Slop score

Each sample receives a score based on human votes:

```text
-1 = definitely slop
 0 = undecided / neutral
+1 = not slop at all
```

The score represents **human judgment**, not an objective measure of quality, AI-generatedness, or factual correctness.

Multiple people may vote on the same sample. This allows agreement and disagreement between annotators to be preserved rather than forcing every sample into a single binary label.

## Data

Each annotation contains information such as:

* `content` — the text being evaluated
* `slop_score` — the submitted human rating
* `source_dataset` — the dataset the sample originated from
* `source_row_id` — the original sample identifier
* `content_hash` — identifier used to help detect duplicate content
* `created_at` — time the annotation was collected

Source samples may contain conversations rather than standalone text.

## Collection

Samples are selected from public datasets and presented randomly to contributors through SlopFinder.

Contributors are given a single continuous slider rather than a list of predefined categories. This is intended to make annotation quick and reduce the friction of contributing.

No account is required to submit a vote.

The dataset is continuously growing as more annotations are collected.

## Intended use

This dataset is intended for:

* training AI-slop classification models
* studying human perception of AI-generated writing
* evaluating whether text classifiers generalize across different sources
* research into stylistic characteristics associated with perceived AI slop

It should **not** be treated as a definitive benchmark for writing quality or AI-generatedness.

## Limitations

The labels represent subjective human judgments. Different contributors may have very different ideas of what constitutes “slop.”

The dataset may also contain biases introduced by:

* the source datasets used for sampling
* the population of contributors
* differences in individual definitions of “slop”
* repeated exposure to similar types of text

A score should therefore be interpreted as **perceived sloppiness**, rather than an objective property of the text.

## Contribute

This project is an early preview, and contributions are welcome.

Try SlopFinder and help expand the dataset:

[Bench Labs Slop Classifier](https://bench-labs.web.app/slopfinder.html)

## License

MIT