README.md

---
title: RM-DETECT
emoji: 🔍
colorFrom: red
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
license: mit
---

# RM-DETECT

**AI 작성 텍스트 탐지 웹 앱 · a transparent, self-hostable AI-vs-human text detector (Korean & English).**

RM-DETECT is an independently built detector that estimates how likely a piece
of text was written by a large language model. It combines **token
predictability** (perplexity) with **stylometric analysis** — both
well-established signals in the AI-text-detection literature — behind a FastAPI
service and a single-page web UI. Paste text → get an overall AI-likelihood
score, a verdict, and a **per-paragraph 의심 영역** view where each paragraph is
colour-graded from green (human) through yellow (mixed) to red (AI), with the
top contributing signals shown on click.

Every score is explainable: the model reports which features pushed a verdict
toward "AI" or "human", so it is not a black box.

> 📐 상세 내부 구조·데이터 흐름·피처 사양은 **[ARCHITECTURE.md](ARCHITECTURE.md)** 참고.

![RM-DETECT UI]({{artifact:0d56d91b-dd73-466b-ae76-aab5421596ed}})

---

## Detection approach

RM-DETECT fuses two independent, complementary signal families into one
classifier:

1. **Token predictability (perplexity).** A small causal language model (GPT-2
   for English, KoGPT2 for Korean) scores how *predictable* each token is. LLMs
   generate text auto-regressively by favouring high-probability next tokens, so
   machine-written text tends to be more predictable — lower perplexity and a
   higher fraction of tokens the model would itself have ranked at the top. This
   is the same principle behind academic detectors such as GLTR and DetectGPT.
2. **Stylometry (model-free).** Sentence-length **burstiness**, logical-connective
   density, punctuation / emoji / informal-marker density (`ㅋㅋ`, `…`, `lol`),
   lexical repetition and entropy, and Korean formal-vs-colloquial register
   endings (`하였습니다` vs `~했음`). Human writing varies far more; AI tends to a
   uniform, formal rhythm.

A logistic-regression classifier fuses **28 features** in total. On the
calibration set it reached **97.2% cross-validated accuracy (AUC 0.994)**. Every
paragraph is scored independently, so mixed human/AI documents are broken down
region by region.

---

## Run it

```bash
# 1. install deps (CPU is fine)
pip install -r requirements.txt

# 2. models — unpack the two language models into ./models/
#    (from models_gpt2_kogpt2.tar.gz), giving ./models/gpt2 and ./models/kogpt2.
#    Or skip this and let transformers pull them from the HF hub on first run.
mkdir -p models && tar -xzf ../models_gpt2_kogpt2.tar.gz -C models

# 3. launch
./run.sh                       # -> http://127.0.0.1:8000
# or:  uvicorn app:app --host 127.0.0.1 --port 8000
```

Open **http://127.0.0.1:8000** in a browser, paste text, and click 분석하기.

> Prefer not to run a server? Open `../rmdetect_demo.html` directly in a browser
> — it's the real UI rendered with a real detection response.

---

## API

**`GET /health`** → `{status, model_loaded, uptime_s, max_chars}`

**`POST /detect`**
```json
// request
{ "text": "검사할 글 ...", "lang": "ko" }   // lang optional: "ko" | "en" | omit for auto
```
```jsonc
// response (abridged)
{
  "language": "ko",
  "overall_ai_probability": 0.50,
  "verdict": "Mixed",
  "n_paragraphs": 3,
  "n_flagged": 1,
  "top_features": [ {"feature": "informal_per100", "contribution": -1.17, "direction": "human"}, ... ],
  "paragraphs": [
    { "index": 0, "start": 0, "end": 40, "text": "...",
      "ai_probability": 0.35, "verdict": "Likely human",
      "flagged": false, "low_confidence": false,
      "top_features": [ ... ] }
  ],
  "elapsed_ms": 2255.8
}
```
`start`/`end` are character offsets into the submitted text, so a client can
highlight the exact source region of each paragraph.

Verdict bands: `≥0.85 AI-generated · ≥0.60 Likely AI · ≥0.40 Mixed · ≥0.15 Likely human · else Human-written`. A paragraph is **flagged** at `ai_probability ≥ 0.60`.

---

## Files

| path | purpose |
|---|---|
| `app.py` | FastAPI backend (`/health`, `/detect`, serves UI) |
| `static/index.html`, `style.css`, `app.js` | single-page UI |
| `aidetect/` | detection engine (perplexity + stylometry) |
| `ai_detector_model.joblib` | fitted classifier |
| `requirements.txt`, `run.sh` | install + launch |

---

## Limitations (MVP)

RM-DETECT is a **research prototype**, not a definitive judgment tool. Carried
over from the underlying engine:

- **Small calibration set (n=36)** and **LLM-imitated "human" samples** — real,
  especially *formal*, human writing (academic papers, polished 자기소개서) will
  over-flag more than the headline accuracy suggests: 인간 글이라도 너무
  형식적이면 오탐지될 수 있음.
- **Small reference LMs** (GPT-2/KoGPT2, ~125M) — a larger or instruction-tuned
  LM would sharpen the perplexity signal.
- **Paraphrase-vulnerable by design** — the detector keys on burstiness and
  informality, so "humanizing" edits (adding colloquialisms, varying sentence
  length) lower the AI score.
- **CPU latency** — ~1–2 s per request on a multi-core machine; noticeably
  slower on shared free-tier CPU. Thresholds should be recalibrated per
  deployment and per acceptable false-positive rate.

Do not use RM-DETECT as sole grounds for a consequential decision (academic
misconduct, hiring). Treat its output as one probabilistic signal among many.