---
language:
  - en
  - es
pipeline_tag: text-classification
tags:
  - ai-generated-text-detection
  - text-classification
  - ensemble
  - deberta
  - lightgbm
  - tf-idf
  - pytorch
  - scikit-learn
---

# EC_MODELS (AI vs Human Detection)

This repository contains **model artifacts** for AI‑generated text detection experiments, including a HOPE classifier and a stacked ensemble (DeBERTa + LightGBM + TF‑IDF/SGD). It is **not a single model**, but a collection of checkpoints and meta‑models produced during experiments.

## Model inventory

Artifacts are stored in `src/ai_vs_human/models/`:

- `hope/`  
  HOPE checkpoints (`fold_*/best_model.pt`) for a transformer‑based classifier with memory modules.
- `deberta_v3_base/`  
  DeBERTa base checkpoints used in the ensemble.
- `lightgbm/`, `lightgbm_numeric/`  
  LightGBM models on engineered features.
- `tfidf_sgd/`  
  TF‑IDF + SGD models.
- `stack_meta/meta_learner.joblib`  
  Logistic‑regression meta‑learner used for stacking.

Out‑of‑fold predictions used to compute metrics and train the stacker are in:

- `src/ai_vs_human/oof/` (e.g., `oof_stack.csv`, `oof_deberta.csv`, `oof_lgb.csv`, `oof_sgd.csv`)

## Intended use

These models are intended for **research and evaluation** of AI‑generated text detection. They can be used to:

- compare HOPE vs. ensemble baselines,
- reproduce experiments from the notebooks,
- evaluate domain‑shift and multilingual robustness.

## How to evaluate (metrics only)

Use the metrics‑only notebook to compute standard metrics without retraining:

- `src/ai_vs_human/metrics_only.ipynb`

This notebook loads `oof_stack.csv` and prints AUC‑ROC, PR‑AUC, Accuracy, Precision, Recall, F1, Brier, and ECE, plus a best‑F1 threshold.

## Training data

Training data and features were produced from the project’s datasets under `src/ai_vs_human/`, including:

- `merged_ai_human_multisocial_features*.csv`

See the dataset‑building and training notebooks for details:

- `src/ai_vs_human/ai_generated_text_detection.ipynb`
- `src/ai_vs_human/hope_train_distributed.py`

## Evaluation notes

Metrics depend on threshold selection and domain. This repo includes tools for:

- internal test evaluation,
- external/ood evaluation,
- calibration and threshold selection.

See:

- `src/ai_vs_human/evaluation_suite.ipynb`

## Limitations and biases

- Performance can degrade under **domain shift** (new sources or languages).
- False‑positive rates for **human multilingual text** can be high without careful calibration.
- These models are **not** a definitive AI‑detection system and should not be used for high‑stakes decisions without additional validation.

## License

No explicit license is included in this repo. Please add a license if you intend to distribute or reuse the models.