rubenfb23 commited on
Commit
3931802
·
verified ·
1 Parent(s): e064b6e

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +87 -3
README.md CHANGED
@@ -1,3 +1,87 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - es
5
+ pipeline_tag: text-classification
6
+ tags:
7
+ - ai-generated-text-detection
8
+ - text-classification
9
+ - ensemble
10
+ - deberta
11
+ - lightgbm
12
+ - tf-idf
13
+ - pytorch
14
+ - scikit-learn
15
+ ---
16
+
17
+ # EC_MODELS (AI vs Human Detection)
18
+
19
+ This repository contains **model artifacts** for AI‑generated text detection experiments, including a HOPE classifier and a stacked ensemble (DeBERTa + LightGBM + TF‑IDF/SGD). It is **not a single model**, but a collection of checkpoints and meta‑models produced during experiments.
20
+
21
+ ## Model inventory
22
+
23
+ Artifacts are stored in `src/ai_vs_human/models/`:
24
+
25
+ - `hope/`
26
+ HOPE checkpoints (`fold_*/best_model.pt`) for a transformer‑based classifier with memory modules.
27
+ - `deberta_v3_base/`
28
+ DeBERTa base checkpoints used in the ensemble.
29
+ - `lightgbm/`, `lightgbm_numeric/`
30
+ LightGBM models on engineered features.
31
+ - `tfidf_sgd/`
32
+ TF‑IDF + SGD models.
33
+ - `stack_meta/meta_learner.joblib`
34
+ Logistic‑regression meta‑learner used for stacking.
35
+
36
+ Out‑of‑fold predictions used to compute metrics and train the stacker are in:
37
+
38
+ - `src/ai_vs_human/oof/` (e.g., `oof_stack.csv`, `oof_deberta.csv`, `oof_lgb.csv`, `oof_sgd.csv`)
39
+
40
+ ## Intended use
41
+
42
+ These models are intended for **research and evaluation** of AI‑generated text detection. They can be used to:
43
+
44
+ - compare HOPE vs. ensemble baselines,
45
+ - reproduce experiments from the notebooks,
46
+ - evaluate domain‑shift and multilingual robustness.
47
+
48
+ ## How to evaluate (metrics only)
49
+
50
+ Use the metrics‑only notebook to compute standard metrics without retraining:
51
+
52
+ - `src/ai_vs_human/metrics_only.ipynb`
53
+
54
+ This notebook loads `oof_stack.csv` and prints AUC‑ROC, PR‑AUC, Accuracy, Precision, Recall, F1, Brier, and ECE, plus a best‑F1 threshold.
55
+
56
+ ## Training data
57
+
58
+ Training data and features were produced from the project’s datasets under `src/ai_vs_human/`, including:
59
+
60
+ - `merged_ai_human_multisocial_features*.csv`
61
+
62
+ See the dataset‑building and training notebooks for details:
63
+
64
+ - `src/ai_vs_human/ai_generated_text_detection.ipynb`
65
+ - `src/ai_vs_human/hope_train_distributed.py`
66
+
67
+ ## Evaluation notes
68
+
69
+ Metrics depend on threshold selection and domain. This repo includes tools for:
70
+
71
+ - internal test evaluation,
72
+ - external/ood evaluation,
73
+ - calibration and threshold selection.
74
+
75
+ See:
76
+
77
+ - `src/ai_vs_human/evaluation_suite.ipynb`
78
+
79
+ ## Limitations and biases
80
+
81
+ - Performance can degrade under **domain shift** (new sources or languages).
82
+ - False‑positive rates for **human multilingual text** can be high without careful calibration.
83
+ - These models are **not** a definitive AI‑detection system and should not be used for high‑stakes decisions without additional validation.
84
+
85
+ ## License
86
+
87
+ No explicit license is included in this repo. Please add a license if you intend to distribute or reuse the models.