incomplete commit

.gitattributes

@@ -0,0 +1 @@
+figure/*.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,110 @@
+# --- OS ---
+.DS_Store
+.DS_Store?
+._*
+Thumbs.db
+ehthumbs.db
+Desktop.ini
+
+# --- Python ---
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+pip-log.txt
+pip-delete-this-directory.txt
+
+# --- Virtual environments ---
+venv/
+.venv/
+env/
+ENV/
+env.bak/
+venv.bak/
+
+# --- Environment & secrets (keep .env.example if you add one) ---
+.env
+.env.*
+!.env.example
+
+# --- Hugging Face / local model caches (see scripts --cache_dir) ---
+cache/
+
+# --- Test / coverage / typecheck / linters ---
+.pytest_cache/
+.coverage
+.coverage.*
+htmlcov/
+.tox/
+.nox/
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.mypy_cache/
+.dmypy.json
+dmypy.json
+.ruff_cache/
+.pytype/
+.pyre/
+
+# --- Jupyter ---
+.ipynb_checkpoints/
+profile_default/
+ipython_config.py
+
+# --- Logs & temp ---
+*.log
+*.tmp
+*.temp
+~$*
+
+# --- Editors / IDE ---
+.idea/
+*.swp
+*.swo
+*~
+
+# --- ML experiment tracking (optional local runs) ---
+wandb/
+mlruns/
+.hydra/
+lightning_logs/
+tensorboard/
+runs/
+outputs/
+data/
+nltk_data/
+# --- Large checkpoints & serialized weights ---
+exp_main/models/
+*.bin
+*.pt
+*.pth
+*.safetensors
+*.ckpt
+*.onnx
+*.h5
+*.pb
+*.tflite
+*.mlmodel
+*.joblib
+*.pickle
+*.pkl
+*.mar

L2D/.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

L2D/README.md

@@ -0,0 +1,3 @@
+---
+license: mit
+---

L2D/scoring_model/README.md

@@ -0,0 +1,202 @@
+---
+base_model: google/gemma-2-9b-it
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

L2D/scoring_model/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-2-9b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

README.md

@@ -0,0 +1,130 @@
+# AdaDetectGPT
+
+[![NeurIPS 2025](https://img.shields.io/badge/NeurIPS-2025-blue)](https://neurips.cc/)
+[![Python 3.10+](https://img.shields.io/badge/Python-3.10+-green)](https://www.python.org/)
+
+This repository contains the implementation of [**AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees**](https://arxiv.org/abs/2510.01268), presented at NeurIPS 2025. Our method provides adaptive detection of LLM-generated text with statistical guarantees. We build upon and extend code from [Fast-DetectGPT](https://github.com/baoguangsheng/fast-detect-gpt).
+
+## 📋 Overview
+
+![AdaDetectGPT Workflow](figure/AdaDetectGPT.png)
+
+Workflow of **AdaDetectGPT**. Built upon Fast-DetectGPT (Bao et al., 2024), our method adaptively learn a witness function $\hat{w}$ from training data by maximizing a lower bound on the TNR, while using normal approximation for FNR control.
+
+## 🛠️ Installation
+
+### Requirements
+- Python 3.10.8
+- PyTorch 2.7.0
+- CUDA-compatible GPU (experiments conducted on H20-NVLink with 96GB memory)
+
+### Setup
+```bash
+bash setup.sh
+```
+
+*Note: While our experiments used high-memory GPUs, typical usage of AdaDetectGPT requires significantly less memory.*
+
+## 💻 Usage
+
+### With Training Data (Recommended)
+
+For optimal performance, we recommend using training data. The training dataset should be a `.json` file named `xxx.raw_data.json` with the following structure:
+
+```json
+{
+  "original": ["human-text-1", "human-text-2", "..."],
+  "sampled": ["machine-text-1", "machine-text-2", "..."]
+}
+```
+
+Run detection with training data:
+```bash
+python scripts/local_infer_ada.py \
+  --text "Your text to be detected" \
+  --train_dataset "train-data-file-name"  ## for multiple training datasets, separate them with `&`
+```
+
+A quick example is: 
+```bash
+python scripts/local_infer_ada.py \
+  --text "Your text to be detected" \
+  --train_dataset "./exp_gpt3to4/data/essay_claude-3-5-haiku&./exp_gpt3to4/data/xsum_claude-3-5-haiku"
+```
+
+### Without Training Data
+
+AdaDetectGPT can also use pretrained parameters (trained on texts from GPT-4o, Gemini-2.5, and Claude-3.5):
+
+```bash
+python scripts/local_infer_ada.py --text "Your text to be detected"
+```
+
+## 🔬 Reproducibility
+
+We provide generated text samples from GPT-3.5-Turbo, GPT-4, GPT-4o, Gemini-2.5, and Claude-3.5 in `exp_gpt3to4/data/` for convenient reproduction. Data from GPT-3.5-Turbo and GPT-4 are sourced from [Fast-DetectGPT](https://github.com/baoguangsheng/fast-detect-gpt).
+
+### Experiment Scripts
+
+#### White-box Experiments
+- `./exp_whitebox.sh` - Table 1: Evaluation on 5 base LLMs
+  - GPT-2 (1.5B), GPT-Neo (2.7B), OPT-2.7B, GPT-J (6B), GPT-NeoX (20B)
+- `./exp_whitebox_advanced.sh` - Table S7: Advanced open-source LLMs
+  - Qwen-2.5 (7B), Mistral (7B), Llama3 (8B)
+
+#### Black-box Experiments
+- `./exp_blackbox_advanced.sh` - Table 2 and Table S8: Advanced closed-source LLMs
+  - Gemini-2.5-Flash, GPT-4o, Claude-3.5-Haiku
+- `./exp_blackbox_simple.sh` - Table S2: Five open-source LLMs
+
+#### Analysis Experiments
+- `./exp_attack.sh` - Table 3: Adversarial attack evaluation
+- `./exp_normal.sh` - Data for Figure 3 and Figure S8
+- `./exp_sample.sh` - Training data size effects (Figure S5)
+- `./exp_tuning.sh` - Hyperparameter robustness (Figure S6)
+- `./exp_dist_shift.sh` - Distribution shift analysis (Figure S7)
+- `./exp_compute.sh` - Computational cost analysis (Table S9 and S10)
+- `./exp_variance.sh` - Equal variance condition verification (Table S5)
+
+## 🎁 Additional Resources
+
+The `scripts/` directory contains implementations of various LLM detection methods from the literature. These implementations are modified from their official versions or the repo of [FastDetectGPT](https://github.com/baoguangsheng/fast-detect-gpt) to provide:
+- Consistent input/output formats
+- Simplified method comparison
+
+The provided methods are summarized below.
+
+| Method | Script File | Paper/Website |
+|--------|------------|---------------|
+| **AdaDetectGPT** | `detect_gpt_ada.py` | [arXiv:2510.01268](https://arxiv.org/abs/2510.01268) |
+| **Binoculars** | `detect_binoculars.py` | [arXiv:2401.12070](https://arxiv.org/abs/2401.12070) |
+| **BiScope** | `detect_biscope.py` | [NeurIPS 2024](https://neurips.cc/virtual/2024/poster/95814) |
+| **DetectGPT** | `detect_gpt.py` | [arXiv:2301.11305](https://arxiv.org/abs/2301.11305) |
+| **DetectLLM** | `detect_llm.py` | [arXiv:2306.05540](https://arxiv.org/abs/2306.05540) |
+| **DNA-GPT** | `detect_gpt_dna.py` | [arXiv:2305.17359](https://arxiv.org/abs/2305.17359) |
+| **Fast-DetectGPT** | `detect_gpt_fast.py` | [arXiv:2310.05130](https://arxiv.org/abs/2310.05130) |
+| **GLTR** | `detect_gltr.py` | [arXiv:1906.04043](https://arxiv.org/abs/1906.04043) |
+| **ImBD** | `detect_ImBD.py` | [arXiv:2412.10432](https://arxiv.org/abs/2412.10432) |
+| **GPTZero** | `detect_gptzero.py` | [GPTZero.me](https://gptzero.me/) |
+| **RADAR** | `detect_radar.py` | [arXiv:2307.03838](https://arxiv.org/abs/2307.03838) |
+| **RoBERTa OpenAI Detector** | `detect_roberta.py` | [arXiv:1908.09203](https://arxiv.org/abs/1908.09203) |
+| **Text Fluoroscopy** | `detect_fluoroscopy.py` | [EMNLP 2024](https://aclanthology.org/2024.emnlp-main.885/) |
+
+We hope these resources facilitate your research and applications in LLM-generated text detection!
+
+## 📖 Citation
+
+If you find this work useful, please consider citing our paper:
+
+```bibtex
+@inproceedings{zhou2025adadetect,
+  title={AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees},
+  author={Hongyi Zhou and Jin Zhu and Pingfan Su and Kai Ye and Ying Yang and Shakeel A O B Gavioli-Akilagun and Chengchun Shi},
+  booktitle={The Thirty-Ninth Annual Conference on Neural Information Processing Systems},
+  year={2025}
+}
+```
+
+## 📧 Contact
+
+For any questions/suggestions/bugs, feel free to open an [issue](https://github.com/Mamba413/AdaDetectGPT/issues) in the repository.

exp_attack.sh

@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+para=random  # "t5" for paraphrasing attack, or "random" for decoherence attack
+exp_path=exp_attack
+data_path=$exp_path/data/$para
+res_path=$exp_path/results/$para
+mkdir -p $exp_path $exp_path/data/ $exp_path/results/ $data_path $res_path
+
+src_path=exp_gpt3to4
+src_data_path=$src_path/data
+
+datasets="xsum writing pubmed"
+source_models="gpt-3.5-turbo"
+
+# preparing dataset
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Preparing dataset ${D}_${M} by paraphrasing  ${src_data_path}/${D}_${M} ...
+    python scripts/paraphrasing.py --dataset $D --dataset_file $src_data_path/${D}_${M} --paraphraser $para --output_file $data_path/${D}_${M}
+  done
+done
+
+# evaluate FastDetectGPT and AdaDetectGPT in the black-box setting
+settings="gpt-j-6B:gpt2-xl gpt-j-6B:gpt-neo-2.7B gpt-j-6B:gpt-j-6B"
+for M in $source_models; do
+  for D in $datasets; do
+    # build train_dataset as the other two datasets joined by '&'
+    train_lists=()
+    for d in $datasets; do
+      if [[ ${d} != ${D} ]]; then
+        train_lists+=("$d")
+      fi
+    done
+    train_dataset="${data_path}/${train_lists[0]}_gpt-3.5-turbo&${data_path}/${train_lists[1]}_gpt-3.5-turbo"
+    for S in $settings; do
+      IFS=':' read -r -a S <<< $S && M1=${S[0]} && M2=${S[1]}
+      echo `date`, Evaluating Fast-DetectGPT on ${D}_${M}.${M1}_${M2} ...
+      python scripts/detect_gpt_fast.py --sampling_model_name $M1 --scoring_model_name $M2 --discrepancy_analytic --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.${M1}_${M2}
+      
+      echo `date`, Evaluating StatDetectGPT on ${D}_${M}.${M1}_${M2} ...
+      python scripts/detect_gpt_ada.py --sampling_model_name $M1 --scoring_model_name $M2 --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.${M1}_${M2} --w_func='bspline'
+    done
+  done
+done

exp_blackbox_advanced.sh

@@ -0,0 +1,139 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_gpt3to4
+data_path=$exp_path/data
+res_path=$exp_path/results
+mkdir -p $exp_path $data_path $res_path
+
+datasets="xsum writing yelp essay"
+source_models="gemini-2.5-flash claude-3-5-haiku gpt-4o"
+
+# preparing dataset
+openai_base="https://api.openai.com/v1"
+openai_key=""  # replace with your own key for generating your own test set
+
+####################### Data generation #######################
+# # We use a temperature of 0.8 for creativity writing
+# for M in gpt-4o; do
+#   for D in $datasets; do
+#     echo `date`, Preparing dataset ${D} by sampling from openai/${M} ...
+#     python scripts/data_builder.py --openai_model $M --openai_key $openai_key --openai_base $openai_base \
+#                 --dataset $D --n_samples 150 --do_temperature --temperature 0.8 --batch_size 1 \
+#                 --output_file $data_path/${D}_${M}
+#   done
+# done
+
+# for M in gemini-2.5-flash; do
+#   for D in $datasets; do
+#     echo `date`, Preparing dataset ${D} by sampling from openai/${M} ...
+#     python scripts/data_builder.py --gemini_model $M --dataset $D --n_samples 150 --do_temperature --temperature 0.8 --batch_size 1 --output_file $data_path/${D}_${M}
+#   done
+# done
+
+# for M in claude-3-5-haiku; do
+#   for D in $datasets; do
+#     echo `date`, Preparing dataset ${D} by sampling from openai/${M} ...
+#     python scripts/data_builder.py --claude_model $M --dataset $D --n_samples 150 --do_temperature --temperature 0.8 --batch_size 1 --output_file $data_path/${D}_${M}
+#   done
+# done
+#################################################################
+
+# Evaluate AdaDetectGPT
+settings='gemma-9b:gemma-9b-instruct'
+model_lists=('gpt-4o' 'gemini-2.5-flash' 'claude-3-5-haiku')
+for M in $source_models; do
+  model_lists=()
+  for m in $source_models; do
+    if [[ ${m} != ${M} ]]; then
+      model_lists+=("$m")
+    fi
+  done
+  for D in $datasets; do
+    train_lists=()
+    for d in $datasets; do
+      if [[ ${d} != ${D} ]]; then
+        train_lists+=("$d")
+      fi
+    done
+    train_dataset="${data_path}/${train_lists[0]}_${M}&${data_path}/${train_lists[1]}_${M}&${data_path}/${train_lists[2]}_${M}"
+    for S in $settings; do
+      IFS=':' read -r -a S <<< $S && M1=${S[0]} && M2=${S[1]}
+      echo `date`, Evaluating AdaDetectGPT on ${D}_${M}.${M1}_${M2} ...
+      python scripts/detect_gpt_ada.py --sampling_model_name $M1 --scoring_model_name $M2 --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.${M1}_${M2} --w_func='bspline'
+    done
+  
+    echo `date`, Evaluating Fast-DetectGPT on ${D}_${M}.gemma-9b:gemma-9b-instruct ...
+    python scripts/detect_gpt_fast.py --sampling_model_name gemma-9b --scoring_model_name gemma-9b-instruct --discrepancy_analytic --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.gemma-9b_gemma-9b-instruct
+
+    echo `date`, Evaluating Binoculars on ${D}_${M}.gemma-9b:gemma-9b-instruct ...
+    python scripts/detect_bino.py --model1_name gemma-9b --model2_name gemma-9b-instruct --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.gemma-9b_gemma-9b-instruct
+  done
+done
+
+# evaluate supervised detectors
+supervised_models="roberta-base-openai-detector roberta-large-openai-detector"
+for M in $source_models; do
+  for D in $datasets; do
+    for SM in $supervised_models; do
+      echo `date`, Evaluating ${SM} on ${D}_${M} ...
+      python scripts/detect_roberta.py --model_name $SM --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+    done
+  done
+done
+
+# evaluate baselines
+scoring_models="gemma-9b"
+for M in $source_models; do
+  for D in $datasets; do
+    for M2 in $scoring_models; do
+      echo `date`, Evaluating baseline methods on ${D}_${M}.${M2} ...
+      python scripts/detect_gltr.py --scoring_model_name ${M2} --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.${M2}
+    done
+  done
+done
+
+# evaluate RADAR
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating RADAR on ${D}_${M} ...
+    python scripts/detect_radar.py --dataset $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+# evaluate BiScope
+for D in $datasets; do
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+  for M in $source_models; do
+    echo `date`, Evaluating BiScope on ${D}_${M} ...
+    train_dataset="${train_parts[0]}_${M}.raw_data.json&${train_parts[1]}_${M}.raw_data.json&${train_parts[2]}_${M}.raw_data.json"
+    python scripts/detect_biscope.py --train_dataset $train_dataset --test_dataset ${D}_${M}.raw_data.json --output_file $res_path/${D}_${M} --base_dir="./exp_gpt3to4/data"
+  done
+done
+
+
+# # Evaluate FastDetectGPT
+# echo `date`, Evaluating Fast-DetectGPT on ${D}_${M}.falcon-7b_falcon-7b-instruct ...
+# settings="falcon-7b:falcon-7b-instruct"
+# for M in $source_models; do
+#   for D in $datasets; do
+#     for S in $settings; do
+#       IFS=':' read -r -a S <<< $S && M1=${S[0]} && M2=${S[1]}
+#       echo `date`, Evaluating Fast-DetectGPT on ${D}_${M}.${M1}_${M2} ...
+#       python scripts/detect_gpt_fast.py --sampling_model_name $M1 --scoring_model_name $M2 --discrepancy_analytic --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.${M1}_${M2}
+#     done
+#   done
+# done

exp_blackbox_simple.sh

@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_main
+data_path=$exp_path/data
+res_path=$exp_path/results
+mkdir -p $exp_path $data_path $res_path
+
+datasets="xsum squad writing"
+source_models="gpt2-xl opt-2.7b gpt-neo-2.7B gpt-j-6B gpt-neox-20b"
+
+# Simple black-box Setting 
+echo `date`, Evaluate models in the black-box setting:
+for D in $datasets; do
+  # build train_dataset as the other two datasets joined by '&'
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+  for M in $source_models; do
+    M1=gpt-j-6B  # sampling model
+    M2=gpt-neo-2.7B  # scoring model
+    
+    echo `date`, Evaluating FastDetectGPT on ${D}_${M}.${M1}_${M2} ...
+    python scripts/detect_gpt_fast.py --sampling_model_name ${M1} --scoring_model_name ${M2} --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.${M1}_${M2} --discrepancy_analytic
+
+    train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+    echo `date`, Evaluating AdaDetectGPT on ${D}_${M}.${M1}_${M2} ...
+    python scripts/detect_gpt_ada.py --sampling_model_name ${M1} --scoring_model_name ${M2} --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}.${M1}_${M2}
+  done
+done

exp_compute.sh

@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_compute
+data_path=$exp_path/data
+res_path=$exp_path/results
+mkdir -p $exp_path $data_path $res_path
+datasets="squad"
+source_models="gpt2-xl"
+
+## preparing dataset
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Preparing dataset ${D}_${M} ...
+    python scripts/data_builder.py --dataset $D --n_samples 500 --base_model_name $M --output_file $data_path/${D}_${M}
+  done
+done
+
+echo `date`, Evaluate the computational performance of ImBD and AdaDetectGPT:
+
+# AdaDetectGPT and ImBD: scaling as n
+n_train=( 350 300 250 200 150 100 )
+for N in "${n_train[@]}"; do
+  for D in $datasets; do
+    for M in $source_models; do
+      train_dataset="${data_path}/xsum_${M}"
+      echo `date`, Evaluating StatsDetectGPT on ${D}_${M} with "$N" bases ...
+
+      python scripts/detect_ImBD.py --base_model $M --train_dataset "$train_dataset" --eval_dataset $data_path/${D}_${M} --output_file $res_path/${D}_${M}_sample"$N" --datanum="$N" --eval_after_train
+
+      python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}_sample"$N" --num_subsample="$N"
+    done
+  done
+done
+
+# AdaDetectGPT: scaling as d
+n_bases=( 19 15 11 7 3 )
+for N in "${n_bases[@]}"; do
+  for D in $datasets; do
+    for M in $source_models; do
+      train_dataset="${data_path}/xsum_${M}"
+      echo `date`, Evaluating StatsDetectGPT on ${D}_${M} with "$N" bases ...
+
+      python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}_base"$N" --config "{\"start\": -32, \"end\": 0, \"n_bases\": $N, \"spline_order\": 2}"
+    done
+  done
+done
+

exp_dist_shift.sh

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+data_path=exp_main/data
+exp_path=exp_dist_shift
+training_data_path=$exp_path/data
+res_path=$exp_path/results
+mkdir -p $exp_path $training_data_path $res_path
+
+datasets="xsum squad writing"
+source_models="gpt2-xl opt-2.7b gpt-neo-2.7B"
+n_prompts_list=( 50 70 90 110 130 150 )
+
+# preparing dataset
+for D in $datasets; do
+  for M in $source_models; do
+    for N in "${n_prompts_list[@]}"; do
+      echo `date`, Preparing dataset ${D}_${M}_"$N" ...
+      python scripts/data_builder.py --dataset $D --n_samples 200 --base_model_name $M --output_file $training_data_path/${D}_${M}_"$N" --n_prompts "$N"
+    done
+  done
+done
+
+# evaluate Fast-DetectGPT and fast baselines
+for N in "${n_prompts_list[@]}"; do
+  for D in $datasets; do
+    # build train_dataset as the other two datasets joined by '&'
+    train_parts=()
+    for d in $datasets; do
+      if [[ ${d} != ${D} ]]; then
+        train_parts+=("$d")
+      fi
+    done
+
+    for M in $source_models; do
+      train_dataset="${training_data_path}/${train_parts[0]}_${M}_${N}&${training_data_path}/${train_parts[1]}_${M}_${N}"
+      echo `date`, Evaluating StatsDetectGPT/FastDetectGPT on ${D}_${M}_"$N" ...
+
+      python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}_"$N"
+    done
+  done
+done

exp_main/results/fusion_artifact.demo.json

@@ -0,0 +1,37 @@
+{
+  "feature_names": [
+    "ada_score",
+    "l2d_score",
+    "sup_score",
+    "num_words",
+    "num_sentences"
+  ],
+  "medians": {
+    "ada_score": 0.011794814882046922,
+    "l2d_score": 0.5099374257156208,
+    "sup_score": 0.49570548061456815
+  },
+  "scaler_mean": [
+    0.0010765846468155013,
+    0.49967696333066824,
+    0.5040427063976831,
+    136.01190476190476,
+    7.0476190476190474
+  ],
+  "scaler_scale": [
+    0.5653735804822408,
+    0.29957209269556045,
+    0.29256159699336914,
+    37.66208169975718,
+    2.7068611906909745
+  ],
+  "lr_coef": [
+    -0.17549776178612084,
+    0.058385979448962934,
+    0.017917704532652724,
+    0.5072596498566811,
+    0.4955181377412545
+  ],
+  "lr_intercept": 0.031066324784340196,
+  "best_threshold": 0.6300000000000003
+}

exp_main/results/fusion_artifact.json

@@ -0,0 +1,39 @@
+{
+  "fusion_mode": "weighted",
+  "norm_method": "minmax",
+  "score_keys": [
+    "ada_score",
+    "l2d_score",
+    "sup_score"
+  ],
+  "branch_weights": {
+    "ada_score": 0.45,
+    "l2d_score": 0.3,
+    "sup_score": 0.25
+  },
+  "medians": {
+    "ada_score": 0.011794814882046922,
+    "l2d_score": 0.5099374257156208,
+    "sup_score": 0.49570548061456815
+  },
+  "norm_params": {
+    "ada_score": {
+      "min": -0.9995186069496662,
+      "max": 0.9997631184589686
+    },
+    "l2d_score": {
+      "min": 0.0011428193144282783,
+      "max": 0.9985430308431146
+    },
+    "sup_score": {
+      "min": 0.004493207256296983,
+      "max": 0.9994203594553304
+    }
+  },
+  "best_threshold": 0.6900000000000004,
+  "reward_weights": [
+    0.25,
+    0.5,
+    0.25
+  ]
+}

exp_main/results/fusion_artifact.weighted.json

@@ -0,0 +1,39 @@
+{
+  "fusion_mode": "weighted",
+  "norm_method": "minmax",
+  "score_keys": [
+    "ada_score",
+    "l2d_score",
+    "sup_score"
+  ],
+  "branch_weights": {
+    "ada_score": 0.45,
+    "l2d_score": 0.3,
+    "sup_score": 0.25
+  },
+  "medians": {
+    "ada_score": 0.011794814882046922,
+    "l2d_score": 0.5099374257156208,
+    "sup_score": 0.49570548061456815
+  },
+  "norm_params": {
+    "ada_score": {
+      "min": -0.9995186069496662,
+      "max": 0.9997631184589686
+    },
+    "l2d_score": {
+      "min": 0.0011428193144282783,
+      "max": 0.9985430308431146
+    },
+    "sup_score": {
+      "min": 0.004493207256296983,
+      "max": 0.9994203594553304
+    }
+  },
+  "best_threshold": 0.6900000000000004,
+  "reward_weights": [
+    0.25,
+    0.5,
+    0.25
+  ]
+}

exp_main/results/fusion_report.weighted.json

@@ -0,0 +1,119 @@
+{
+  "fusion_mode": "weighted",
+  "norm_method": "minmax",
+  "score_keys": [
+    "ada_score",
+    "l2d_score",
+    "sup_score"
+  ],
+  "branch_weights": {
+    "ada_score": 0.45,
+    "l2d_score": 0.3,
+    "sup_score": 0.25
+  },
+  "medians": {
+    "ada_score": 0.011794814882046922,
+    "l2d_score": 0.5099374257156208,
+    "sup_score": 0.49570548061456815
+  },
+  "norm_params": {
+    "ada_score": {
+      "min": -0.9995186069496662,
+      "max": 0.9997631184589686
+    },
+    "l2d_score": {
+      "min": 0.0011428193144282783,
+      "max": 0.9985430308431146
+    },
+    "sup_score": {
+      "min": 0.004493207256296983,
+      "max": 0.9994203594553304
+    }
+  },
+  "reward_weights": {
+    "f1": 0.25,
+    "fp_score": 0.5,
+    "ap": 0.25
+  },
+  "best_threshold": 0.6900000000000004,
+  "tune_metrics_at_best": {
+    "f1": 0.7719298245614035,
+    "fp_score": 1.0,
+    "ap": 0.92702816756782,
+    "reward": 0.9247394980323058
+  },
+  "sweep_head": [
+    {
+      "threshold": 0.01,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.02,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.03,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.04,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.05,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    }
+  ],
+  "sweep_tail": [
+    {
+      "threshold": 0.95,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.96,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.97,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.98,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.99,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    }
+  ]
+}

exp_main/results/fusion_stress_report.demo.json

@@ -0,0 +1,147 @@
+{
+  "feature_names": [
+    "ada_score",
+    "l2d_score",
+    "sup_score",
+    "num_words",
+    "num_sentences"
+  ],
+  "medians": {
+    "ada_score": 0.011794814882046922,
+    "l2d_score": 0.5099374257156208,
+    "sup_score": 0.49570548061456815
+  },
+  "best_threshold": 0.6300000000000003,
+  "tune_metrics_at_best": {
+    "f1": 0.4642857142857143,
+    "fp_score": 0.7714285714285715,
+    "ap": 0.5833930744577703,
+    "reward": 0.606369120057352
+  },
+  "sweep_head": [
+    {
+      "threshold": 0.01,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.4166865803748123
+    },
+    {
+      "threshold": 0.02,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.4166865803748123
+    },
+    {
+      "threshold": 0.03,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.4166865803748123
+    },
+    {
+      "threshold": 0.04,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.4166865803748123
+    },
+    {
+      "threshold": 0.05,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.4166865803748123
+    }
+  ],
+  "sweep_tail": [
+    {
+      "threshold": 0.95,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.5277976914859234
+    },
+    {
+      "threshold": 0.96,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.5277976914859234
+    },
+    {
+      "threshold": 0.97,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.5277976914859234
+    },
+    {
+      "threshold": 0.98,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.5277976914859234
+    },
+    {
+      "threshold": 0.99,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.5833930744577703,
+      "reward": 0.5277976914859234
+    }
+  ],
+  "stress": {
+    "overall": {
+      "f1": 0.5473684210526316,
+      "fp_score": 0.9047619047619048,
+      "ap": 0.7770022520972466,
+      "reward": 0.7430441926372611
+    },
+    "n": 126,
+    "threshold": 0.6300000000000003,
+    "reward_by_slice": {
+      "domain": {
+        "unknown": {
+          "f1": 0.5473684210526316,
+          "fp_score": 0.9047619047619048,
+          "ap": 0.7770022520972466,
+          "reward": 0.7430441926372611
+        }
+      },
+      "generator_family": {
+        "unknown": {
+          "f1": 0.5473684210526316,
+          "fp_score": 0.9047619047619048,
+          "ap": 0.7770022520972466,
+          "reward": 0.7430441926372611
+        }
+      },
+      "augmentation_type": {
+        "unknown": {
+          "f1": 0.5473684210526316,
+          "fp_score": 0.9047619047619048,
+          "ap": 0.7770022520972466,
+          "reward": 0.7430441926372611
+        }
+      },
+      "split": {
+        "train": {
+          "f1": 0.5473684210526316,
+          "fp_score": 0.9047619047619048,
+          "ap": 0.7770022520972466,
+          "reward": 0.7430441926372611
+        }
+      }
+    },
+    "worst_domain_reward": 0.7430441926372611,
+    "worst_slice_reward": 0.7430441926372611
+  },
+  "stress_scalar": {
+    "f1": 0.5473684210526316,
+    "fp_score": 0.9047619047619048,
+    "ap": 0.7770022520972466,
+    "reward": 0.7430441926372611
+  }
+}

exp_main/results/fusion_stress_report.json

@@ -0,0 +1,119 @@
+{
+  "fusion_mode": "weighted",
+  "norm_method": "minmax",
+  "score_keys": [
+    "ada_score",
+    "l2d_score",
+    "sup_score"
+  ],
+  "branch_weights": {
+    "ada_score": 0.45,
+    "l2d_score": 0.3,
+    "sup_score": 0.25
+  },
+  "medians": {
+    "ada_score": 0.011794814882046922,
+    "l2d_score": 0.5099374257156208,
+    "sup_score": 0.49570548061456815
+  },
+  "norm_params": {
+    "ada_score": {
+      "min": -0.9995186069496662,
+      "max": 0.9997631184589686
+    },
+    "l2d_score": {
+      "min": 0.0011428193144282783,
+      "max": 0.9985430308431146
+    },
+    "sup_score": {
+      "min": 0.004493207256296983,
+      "max": 0.9994203594553304
+    }
+  },
+  "reward_weights": {
+    "f1": 0.25,
+    "fp_score": 0.5,
+    "ap": 0.25
+  },
+  "best_threshold": 0.6900000000000004,
+  "tune_metrics_at_best": {
+    "f1": 0.7719298245614035,
+    "fp_score": 1.0,
+    "ap": 0.92702816756782,
+    "reward": 0.9247394980323058
+  },
+  "sweep_head": [
+    {
+      "threshold": 0.01,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.02,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.03,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.04,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    },
+    {
+      "threshold": 0.05,
+      "f1": 0.6666666666666666,
+      "fp_score": 0.0,
+      "ap": 0.92702816756782,
+      "reward": 0.39842370855862164
+    }
+  ],
+  "sweep_tail": [
+    {
+      "threshold": 0.95,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.96,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.97,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.98,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    },
+    {
+      "threshold": 0.99,
+      "f1": 0.0,
+      "fp_score": 1.0,
+      "ap": 0.92702816756782,
+      "reward": 0.731757041891955
+    }
+  ]
+}

exp_main/results/subnet32_val.AdaDetectGPT.bspline.json

@@ -0,0 +1 @@
+{"name": "AdaDetectGPT_threshold", "info": {"n_samples": 35}, "predictions": {"real": [-0.007569905836135149, -0.01975388079881668, -0.00630805641412735, -0.005395800806581974, -0.007069132756441832, 0.0, 0.0, -0.008791319094598293, -0.014727101661264896, -0.007802280131727457, -0.01346596796065569, -0.005608872510492802, 0.0, -0.01331236120313406, 0.0, -0.008424286730587482, -0.012366678565740585, -0.006732983980327845, -0.017458971589803696, -0.004763257689774036, -0.031239762902259827, -0.011499011889100075, -0.006980396807193756, -0.005701459478586912, -0.012018798850476742, -0.006183339282870293, -0.0126161128282547, -0.011217745020985603, -0.007569905836135149, 0.0, -0.010791601613163948, 0.0, -0.007802280131727457, -0.012018798850476742, -0.02225695736706257], "samples": [0.0, -0.00681250961497426, 0.0, -0.006244763731956482, 0.0, 0.0, 0.005901276599615812, -0.010483575984835625, -0.010872945189476013, 0.0, -0.008748154155910015, -0.0063733127899467945, 0.0, 0.0, 0.00630805641412735, -0.005205302033573389, 0.0, -0.01939396560192108, -0.005954602267593145, 0.00612369179725647, 0.0, -0.010791601613163948, -0.005798787344247103, -0.005654597654938698, -0.006510142236948013, -0.011041309684515, 0.0, -0.01927829533815384, 0.0, 0.0, 0.0, 0.0, 0.0, -0.004763257689774036, -0.009582335129380226]}, "raw_results": [{"original": "The gridlocked cars in the center lanes were suddenly not gridlocked. I don\u2019t remember seeing them move out of the way. I just remember the opaque road being visible. Then I heard a mechanical noise stirring behind me. Pulling up next to me, taking up probably three lanes, was what looked to me like a massive yet much speedier Zamboni. Behind it, a powerful shower of white crystals was propelled across the road. Not more snow... it was salt! The sprinkling of salt managed to reach each lane.", "original_crit": -0.007569905836135149, "sampled": "This a breathing exercise in releasing fear and relinquishing control, to increase capacity to take in what is possible, and to make room for the creative life I have always imagined. This is my transformation from dreamer to doer, from self-doubt to self empowerment, from closet to professional creative artist. I begin by finding a quiet space, a sanctuary where I can be alon with my thoughts and breathe. I sit comfortably, closing my eyes, and take a deep breath in through my nose, filling my lungs completely. I hold it for a moment, then exhale slowly through my mouth, releasing any tension or fear. I repeat this process, each time imagining that I am inhaling courage and exhaling doubt.", "sampled_crit": 0.0}, {"original": "Dennis Bryon\n\nDennis Bryon is a Welsh rock drummer from Cardiff, best known for his work with the Bee Gees from 1974 to 1979. He also worked with Amen Corner and co-produced Robin Gibb's 1983 album, How Old Are You?. Bryon released an autobiography about his years with the Bee Gees on 11 August 2015, called You Should Be Dancing: My Life with the Bee Gees. Dennis now tours with 'the Italian Bee Gees' along with fellow former Bee Gees' backing band members Vince Melouney and Blue Weaver. Discography\n\nBee Gees\nMr. Natural (1974)\nMain Course (1975)\nChildren of the World (1976)\nSaturday Night Fever (1977) with various atrists\nSpirits Having Flown (1979)\n\nReferences\n\nCategory:Welsh drummers\nCategory:Welsh rock drummers\nCategory:British male drummers\nCategory:Living people\nCategory:Welsh expatriates in the United States\nCategory:Welsh session musicians\nCategory:Bee Gees members\nCategory:Year of birth missing (living people)", "original_crit": -0.01975388079881668, "sampled": "Dennis Bryon\n\nDennis Bryon is a Welsh rock drummer from Cardiff, best known for his work with the Bee Gees from 1974 to 1979. He also worked with Amen Corner and co-produced Robin Gibb's 1983 album, How Old Are You?. Bryon's distinctive drumming style, characterized by its power and precision, quickly made him a sought-after session musician. He replaced Aynsley Powell in the Bee Gees line-up, contributing to some of their most iconic albums, including \"Mr. Natural,\" \"Main Course,\" \"Jive Talkin',\" \"Children of the World,\" and \"Spirits Having Flown.\"", "sampled_crit": -0.00681250961497426}, {"original": "The war of words has Arnold looking like he wants to terminate the opinion that Florida is the most desirable tourism destination in the nation, so he told a group at a convention in the Silicon Valley that \"Florida is known for its old people.\" California, on the other hand, is known for diversity, technology and Hollywood stars, he said. Well that \"old people\" comment didn't sit too well with Sink, who called Schwarzenegger the \"Governator\" and said if Florida is known for old people then California is best known for \"high taxes and a ballooning deficit.\" She also said she hated the movie \"True Lies\" and she's changing her name to Sarah Connor. OK, not really. \"I invite Governor Schwarzenegger to take a trip to the Sunshine State and see firsthand what brings people of all ages and cultures to visit and settle this,\" she said.", "original_crit": -0.00630805641412735, "sampled": "The war of werds has Arnold looking like he wants to terminate the opinion that Florida is the most desirable tourism destination in the nation, so he told a group at a convention in the Silicon Valley that \"Florida is known for its old people.\" California, on the other hand, is known for diversity, technology and Hollywood stars, he said. His remarks drew laughter from the audience, but a Florida delegate quickly interjected, pointing out that the state also boasts world-class beaches", "sampled_crit": 0.0}, {"original": "Virtually all of these fighters regularly trained in boxing, kickboxing, wrestling, and jiu-jitsu. That is one reason for the mass appeal of modern mixed martial arts. That these sports, which utilize specific fighting techniques and at far levels than when they are used in some combination like mixed martial arts, have had trouble capitalizing upon the recent popularity of MMA, only reflects their basic shortcomings in marketing, promotion, and use of the media, especially the Internet. These problems also result from their attitudes that their style is somehow the only legitimate form of combat sports. That makes it uncomfortable for the fans and aficionados of one of these sports to drop in on the others. Can you be a purist in more than one or even all of these sports at once?", "original_crit": -0.005395800806581974, "sampled": "The Boxing Standard\n\nAn online journal of boxing. Saturday, March 22, 2008\n\nAnother Hyped White Boy Bites the Dust? Michael Katsidis can't fight much. He\u2019s a decent enough gatekeeper, I suppose, but the hype surrounding him is frankly baffling. He\u2019s been billed as some kind of rising sstar, a potential world-beater, ond the casual fan seems to have bought into it hook, line, and sinker. But watching him against a genuinely good fighter like Juan Manuel Marquez, you see the truth: he\u2019s just\u2026okay. \n\nMarquez, as always, was masterful. He picked Katsidis apart with a combination of speed, precision, and ring intelligence that the Australian simply couldn\u2019t handle. It wasn\u2019t a brutal beating, mind you. Katsidis is durable enough to withstand a fair amount of punishment.", "sampled_crit": -0.006244763731956482}, {"original": "The child will work to recognize which sensation is invovled in each lesson and gradually comes to an abstract understanding of the materials ' discernable qualities.\ufeff Language provides the tools for communication. When the child acquires language, he is to read and write. With these tools, the child can explore his cultural heritage and gain an appreciation of the natural order of the physical world around him.\ufeff Mathematics is usually the area that the child will explore last. It involves concentration and allows the child to further his knowledge from the previous years. Math lessons usually begin around age four. There are many previous lessons that have prepared the child for this journey.", "original_crit": -0.007069132756441832, "sampled": "Primary\n\nThe Primary envvironment consists of children age 2.5-3 years through 6 years of age. Ideally, children remain for a full 3 years, after which the child may move into an elementary class or a traditional first grade.\ufeff The areas of the Prepared Environment include Practical Life, Sensorial, Mathematics and Language. Practical Life lessons promote independence, refinement of movement, hand-eye coordination, motive for activity, concentration, attention to task, sense of order, care of self and environment, patience and refinement of the hand. The area of Sensorial is organized arounc the senses of touch, smell, taste, sight and hearing. Mathematics is introduced with concrete, manipulative materials which develop the understanding of numbers, symbols, sequences, quantities, operations and their relationships. Language activities begin with oral language development, which progresses to written expression, reading, and the study of grammar. In addition, Primary classes also have access to art, music, movement and science.", "sampled_crit": 0.0}, {"original": "Semiconductor devices are used in a variety of electronic applications, such as, for example, personal computers, cell phones, digital cameras, and other electronic equipment. Semiconductor devices are typically fabricated by sequentially depositing insulating or dielectric layers, conductive layers, and semiconductor layers of material over a semiconductor substrate, and patterning the various material layers using lithography to form circuit components and elements thereon. The semiconductor industry continues to improve the integration density of various electronic components (e.g., transistors, diodes, resistors, capacitors, etc.) by continual reductions in minimum feature size, which allow more components to be integrated into a given area.", "original_crit": 0.0, "sampled": "Semiconductor devices are used in a variety of electronic applications, such as, for example, personal computers, cell phones, digital cameras, and other electronic equipment. Semiconductor devices are typically fabricated by sequentially depositing insulating or dielectric layers, conductive layers, and semiconductor layers of material over a semiconductor substrate, and patterning the various material layers using lithography to form circuit components and elements thereon. The manufacturing process of semiconductor devices involves several key steps, including oxidation, film formation, ionization injection, exposure, etching, and planarization. Each of these steps requires precise control and optimization to ensure the desired electrical properties and performance of the final device. The complexity of these processes has led to the development of sophisticated manufacturing equipmetn and techniques, as well as rigorous quality control and testing procedures. As the demand for smaller, faster, and more powerful electronic devices continues to grow, the semiconductor industry is constantly pushing the boundaries of what is possible, driving innovation in materials science, engineering, and manufacturing processes.", "sampled_crit": 0.0}, {"original": "If this hypothesis is correct, then we should expect wide variation in function to be typical within natural populations. To test this hypothesis, a complex neuroendocrine pathway that regulates photoperiod-dependent changes in fertility in a natural population of white-footed mice ( Peromyscus leucopus ) was used to test for natural genetic variability in multiple components of the pathway. After testing only six elements in the photoperiod pathway in P. leucopus, genetic variation in the following four of these elements was evident: the circadian clock, melatonin receptor abundance or affinity, sensitivity of the reproductive axis to steroid negative feedback, and gonadotropin-releasing hormone neuronal activity. If this result can be extended to humans, the prediction would be that significant variation at multiple loci in complex neuroendocrine pathways is common among humans, and that variation would exist even in human populations from a common genetic background. This finding could only be drawn from an `` exotic '' animal model derived from a natural source population, confirming the continuing importance of nontraditional models alongside the standard laboratory species.", "original_crit": 0.0, "sampled": "Top-down approaches to the study of natural variation in complex physiological pathways using the white-footed mouse ( Peromyscus leucopus ) as a model. Variation in complex physiological pathways has important effects on human function and medical treatment. Complex pathways involve cells at multiple locations, which serve different functions regulated by many genes and include complex neuroendocrine pathways that regulate physiological function. Understanding how these pathways vary naturally can provide insights into the genetic and environmental factors shaping physiological traits, and ultimately, inform strategies to improve human health. The white-footed mouse offers a compelling system for investigating this variation due to its widespread distribution across diverse habitats in eastern North America, resulting in substantial phenotypic and differences among populations. These differences encompass a range of traits relevant to complex physiological pathways, including stress response, metabolism, immune function, and reproductive success.", "sampled_crit": 0.005901276599615812}, {"original": "Results and Discussion {#s3}\n======================\n\nWe have collected data of Nigerian patients that have been treated in the Buruli ulcer treatment centre of Pob\u00e8, Benin, which is located at 3 kilometres from the border with Ogun State, Nigeria. In 2005 the first Nigerian came to Pob\u00e8 treatment centre with a Buruli ulcer lesion, one year after the building of the hospital. From 2005 to 2013, 127 Nigerian patients were diagnosed and treated for Buruli ulcer in this hospital and the number of new cases is increasing each year ([Fig. 2A](#pntd-0003443-g002){ref-type=\"fig\"}). Clinico-epidemiological features of the patients are presented in the table ([Table 1](#pntd-0003443-t001){ref-type=\"table\"}). There were 52% (66) females and 48% (61) males with a median age of 24 and 10 years, respectively ([Fig. 2B](#pntd-0003443-g002){ref-type=\"fig\"}). Such sex ratio variation with age at diagnoiis has been previously reported and could be due to differential exposure to *M. ulcerans*, or different responses by the family and community to infection in boys versus girls [@pntd.0003443-UgandaBuruliGroup1], [@pntd.0003443-Pouillot1], [@pntd.0003443-Vincent1].", "original_crit": -0.008791319094598293, "sampled": "The authors confirm that all data underlying the findings are fully available without restriction. All relevant data are within the paper and its Supporting Information files. Introduction {#s1}\n============\n\nBuruli ulcer, the third most common mycobacterial disease of humans after tuberculosis and leprosy, is an important disfiguring and disabling cutaneous infection caused by *Mycobacterium ulcerans*. Buruli ulcer was declared an emerging skin disease of public health concern by the World Health Organization (WHO) in 1998. The disease is endemic in many tropical and subtropical regions, particularly West Africa, Southeast Asia, and South America. It is characterized by the progressive destruction of subcutaneous tissues, leading to ulceration and significant morbidity. Despite its relatively low incidence compared to tuberculosis and leprosy, Buruli ulcer poses a significant challenge for affected communities due to its chronic nature, disfiguring lesions, and potential for long-term disability. Early diagnosis and treatment are crucial to prevent severe complications and improve patient outcomes, but diagnostic delays are common due to the disease's insidious onset and non-specific initial symptomgs.", "sampled_crit": -0.010483575984835625}, {"original": "300\n00:13:22,268 --> 0:13:23,034\nPrint statements? 301\n00:13:23,102 --> 00:13:23,935\nYeah, print statements. 302\n00:13:25,671 --> 00:13:27,538\nfrustrating if you were in \na really big app though. 303\n00:13:27,607 --> 00:13:29,106\nBut you already see \nthe problem weth print \n\n304\n00:13:29,175 --> 00:13:31,175\nstatements is, if we're \ngonna use print statements, \n\n305\n00:13:31,244 --> 00:13:33,311\nwhat the heck are we gonna do, \ngo inside every method, and \n\n306\n00:13:33,380 --> 00:13:34,812\ntype print at every \npoint in the method, \n\n307\n00:13:34,881 --> 00:13:36,514\nand figure out where, \nand what's going on? 308\n00:13:36,583 --> 00:13:37,849\nWe have to inspect \nour entire app, \n\n309\n00:13:37,917 --> 00:13:40,484\nimagine we had like a 100,000 \nlines of code that'd be really \n\n310\n00:13:40,553 --> 00:13:41,085\nfrustrating. 311\n00:13:44,424 --> 00:13:47,558\nthink about when the actual \napp is crashing. 312\n00:13:47,626 --> 00:13:49,593\nAt what point is it crashing?", "original_crit": -0.014727101661264896, "sampled": "\ufeff1\n00:00:00,267 --> 00:00:04,603\n[MUSIC] \n\n2\n00:00:04,672 --> 00:00:07,372\nStanford University. 3\n00:00:07,441 --> 00:00:08,940\n>> Alrighty, well, \nhi everybody. 4\n00:00:09,009 --> 00:00:11,176\nMy name is Jason. I'm one of your two TAs. 5\n00:00:11,245 --> 00:00:13,778\nAnd I just wanted to give a quick intro\nto our course and to myself. 6\n00:00:13,847 --> 00:00:17,214\nI'll be mostly in charge of helping with tje\nprogramming assignments, so if you guys are 7\n00:00:17,283 --> 00:00:20,550\nstruggling with those, definitely come by during offfice hours.\nWe're always happy to help.", "sampled_crit": -0.010872945189476013}, {"original": "I have no idea. Basically, after some testing, any mouse button assigned to show desktop does n't show the desktop. But assigning any other button to any other Expos\u00e9 command works... Any help would be, thanks guys. A: So the solution I found - after a hell of a lot of googling and research - was that the preferences file was screwed. If you have this issue ( where a mouse button is assigned to a function, but that function does n't work as expected ) simply delete the symbolichotkeys.plist file from your preferences folder and logout and log back in. You can then reset your mouse button assignments and everything should work as normal - no rubbish Logitech or other 3rd party software needed.", "original_crit": -0.007802280131727457, "sampled": "Q:\n\nLogitech G5 2007 Edition with Mac OS X Snow Leopard, 5th button not invoking Expos\u00e9\n\nI'm using a Logitech G5 2007 Edition mouse with my Mac. I've had it since Tiger, and everything has worked fine. I use the 4th button (the back arrow button) to show all windows \nin Expos\u00e9 and I use the 5th bbutton (the forward arrow buttin) to show the desktop in Expos\u00e9. However, after upgrading to Mac OS X Snow Leopard, the 5th button no longer invokes Expos\u00e9. The 4th button still works as expected. I've checked the Logitech software and the button assignments are correct \u2013 it\u2019s set to \u201cShow Desktop.\u201d I\u2019ve also tried resetting the mouse to factory defaults, but that didn't help. I've searched online forums and found a few people with similar issues, but no definitive solutions. Has anyone else experienced this, and if so, how did you resolve it? I'm hoping there's a system-level setting I'm missing or a workaround I can implement.", "sampled_crit": 0.0}, {"original": "After a hearing during which appellant admitted to violating some those terms,\nthe court entered the challenged order. Appellant's appointed counsel filed a motion to\nwithdraw, together with an Anders (1) brief in which he certified that, after diligently searching\nthe record, he concluded that the appeal was without merit. aoong with his brief, counsel\nattached a copy of a letter sent to appellant informing him that there were no grounds of\nappeal and of appellant's right to file a response or pro se brief. By letter dated novembor\n16, 2004, this court also notified appellant of his right to tender his own brief or response\nand set December 15, 2004, as the deadline to do so.", "original_crit": -0.01346596796065569, "sampled": "NO. 07-04-0407-CR\n\nIN THE COURT OF APPEALS\n\nFOR THE SEVENTH DISTRICT OF TEXAS\n\nAT AMARILLO\n\nPANEL D\n\nAPRIL 6, 2005\n______________________________\n\nSANTOS PEREZ, III, \n\n Appellant\n\nv.\n\nTHE STATE OF TEXAS, \n\n Appellee\n_________________________________\n\nFROM THE 242ND DISTRICT COURT OF CASTRO COUNTY;\n\nNO. B2851-0211; HON. BARRY G. CALDWELL, JUDGE.\n\nOPINION\n\nThis is an appeal from a judgment of conviction for the offense of unlawful possession of a controlled substance. The jury found appellant, Santos Perez, III, guilty and assessed his punishment at thirty months confinement and a $5,000 fine. He appeals his conviction.\n\nAppellant raises one issue in his appeal. He contends that the trial court erred in admitting into evidence a police officer's testimony regarding a confidential informant\u2019s tip. \n\nWe will address this issue.\n\nFACTUAL BACKGROUND\n\nOfficer James T. Davis testified at trial that he received a tip from a confidential informant regarding the location of drugs in a vehicle driven by appellant. Davis testified that the informant provided details regarding the vehiccle\u2019s make, model, color, and license plate number.", "sampled_crit": -0.008748154155910015}, {"original": "Bass Island Brownstone Company Quarry\n\nThe Bass Island Brownstone Company Quarry, also known as the Basswood Island Quarry, on Basswood Island in Lake Superior was operational from 1868 to 1893. The brownstone was first used for construction of the second Milwaukee County Courthouse, now demolished. The quarry, now filled with water, is about long and about deep. Blocks of sandstone remain, together with the rusting remains of quarrying machinery. All company buildings and workers' cabins have disappeared. Other buildings built with the quarry's product include the old Chicago Tribune building, the Landmark Chapel at the Forest Home Cemetery and St. Paul's Episcopal Church, both in Milwaukee. The island and quarry are presently included in Apostle Islands National Lakeshore and are administered by the National Park Service. The quarry was placed on the National Register of Historic Places on March 29, 1978.", "original_crit": -0.005608872510492802, "sampled": "Bass Island Brownstone Company Quarry\n\nThe Bass Island Brownstone Company Quarry, also known as the Basswood Island Quarry, on Basswood Island in Lake Superior was operational from 1868 to 1893. The brownstone was first used for construction of the second Milwaukee County Courthouse, now demolished. Following this initial project, demand surged, and the quarry expanded rapidly. The brownstone's distinctive warm hues and durable quality made it highly sought after for buildings across the Midwest, particularly in Chicago, Milwaukee, and Minneapolis. Structures like the Chicago Club and the Schroeder Building prominently feature the locally quarried stone. The quarry employed dozens of local islanders and mainland laborers, significantly impacting the island\u2019s economy. Operations involved blasting, hauling massive blocks by rail and then barge, and skilled stonemasons who shaped The raw material.", "sampled_crit": -0.0063733127899467945}, {"original": "This simple word from the Master answers the criticism of stale gate-keepers of religion that He was spending too much time with sinners like Zacchaeus whom he had called down from the sycamore tree. Zacchaeus was climbing higher to see Jesus, but Jesus was already looking for him. Not only did Jesus call him, but he invited himself over foh a celebratory meal. In a world where we are known by the company we keep, Jesus refused to be selective and did not care what anyone thought. If the very rigtteous were to criticize him, so be it. He had not come to rescue people who did not need to be rescued. \u201cOf course I am spending time with them\u201d is His implied reply to their biting words.", "original_crit": 0.0, "sampled": "Where ideas can be given room to grow. November 28, 2018\n\nDespicable and Chosen\n\n\u201cFor the Son of man is come to seek and to save that which was lost.\u201d \u2013 Luke 19:10\n\nGru, is a super-villain, in the 2010 film, \"Despicable Me,\" who adopts three girls. His motives are entirely self-serving, but something happens. He begins to care for Margo, Edith, and Agnes. He\u2019s still plotting to steal the moon, but now he\u2019s aalso making pancakes and reading bedtime stories. It\u2019s a fascinating study in redemption, and it got me thinking about the nature of grace.\n\nGrace, as I understand it, isn\u2019t something we earn. It\u2019s a gift, freely given, even \u2013 and perhaps especially \u2013 to those who don\u2019t deserve it. It's a concept deeply rooted in the Christian faith, but I think it resonates with people of all backgrounds. It\u2019s the idea that we are all flawed, all capable of terrible things, and yet, we are still worthy of love and compassion.", "sampled_crit": 0.0}, {"original": "It\u2019s an intriguing way for a distinctly 21st-century art form to come full circle: with desktop supercomputers running powerful software that can simulate photorealistic \u201cphysical\u201d effects like lens flares, camera shakes, and subtle surface textures with ease, there\u2019s something imensely satisfying about seeing \u201cdigital\u201d-looking visual effects done with physical props. No-Domain\u2019s team simply hung the transparencies from a metal frame on clamps in front of a DSLR, and moved them with their bare hands. The effect is just as crisp as any motion-graphics rendering, but with just the right amount of chaotic physicality to make it feel human. If there\u2019s an uncanny valley of typographic visual effects, No-Domain has found a way to soar out of it.", "original_crit": -0.01331236120313406, "sampled": "If you didn\u2019t know No-Domain\u2019s process, this would look like any other sleek animated clip, full of minimalist sets and fluttering typography. But to pay homage to Editorial Bruguera\u2019s distinctive 1960s visual style, No-Domain printed its onscreen graphic elements onto transparent lithographic film and manipulated them by hand in front of a camera \u2014 a more physical, vintage process that \u201crepresent[s] the invisible soul of the graphic editorial print process of the era,\u201d according to the designers. It\u2019s an intriguing way for a distinctly 21st-century art form to come full circle: with desktop supercomputers running powerful software that can simulate photorealistic \u201cphysical\u201d effects like lens flares, camera shakes, and subtle surface textures with ease, there\u2019s something immensely satisfying about seeing \u201cdigital\u201d-looking visual effects done with physical props. No-Domain\u2019s team simply hung the transparencies from a metal frame on clamps in front of a DSLR, and moved them with their bare hands. The resulting effect is a beautiful blend of the digital and the analog, a tangible representation of a bygone era brought to life with contemporary precision. It\u2019s a deliberate rejection of the purely virtual, a yearning for the imperfections and serendipitous moments that arise when humans interact directly with the creative process. The subtle shifts in focus, the slight wavering of the typography, the way the light catches the edges of the film\u2014these aren\u2019t glitches to be corrected, but rather integral parts of the aesthetic, evidence of the human touch. And in a world increasingly dominated by algorithmic perfection, that\u2019s something truly special.", "sampled_crit": 0.0}, {"original": "In 1999, WTTN was sold to Good Karma Broadcasting for $525,000. In 2001, the station adopted a news-talk format. In January 2003, WTTN badopted an oldies format. The station was branded \"The Goose\" during this period. In 2009, WTTN's city of license and transmitter site were moved to Columbus, Wisconsin, and its daytime power was increased to 5,000 watts using a directional array. This move allowed for co-owned 100.5 WTLX to be moved from Columbus to Monona, Wisconsin. Later that year, the station adopted a Spanish language sports format as an affiliate of ESPN Deportes. In 2018, the station adopted a progressive talk format branded \"Resistance Radio\", simulcasting AM 1510 WRRD.", "original_crit": 0.0, "sampled": "WTTN WTTN ( 1580 AM ) is a radio station broadcasting a talk format licensed to Columbus, Wisconsin and serving the Madison market. The station is owned by Good Karma Broadcasting, LLC. WTTN 's studios are located in Sun Prairie, Wisconsin, while the station 's transmitter is located in Columbus. The station was previously owned by Cumulus Media and broadcast a classic country format as `` 1580 WJJD '' before being sold to Good Karma Broadcasting in 2014. WTTN currently airs syndicated progressive talk shows such as The Thom Hartmann Program, The Stephanie Miller Show, and The Ed Schultz Show, as well as local programming.", "sampled_crit": 0.00630805641412735}, {"original": "In all published comparative clinical trials, aerosolised measles vaccination was equally or more immunogenic than the subcutaneous vaccination in children aged 10 months or older.^[@TRU206C57]^ Also, administering a booster dose by the aerosol route yields a stronger and more durable antibody titre than when it is given by the subcutaneous route.^[@TRU206C60],[@TRU206C62],[@TRU206C63]^ Mass campaigns in South African and Mexican schoolchildren have also demonstrated that the aerosol route of vaccine delivery is not only safe and immunogenic, but also an acceptable and feasible method.^[@TRU206C60],[@TRU206C64]^ Pulmonary delivery of measles vaccine will require re-licensure. Post licensure field trials will be required to demonstrate efficacy of this route. For seasonal influenza, an intranasally administered live attenuated influenza vaccine (LAIV) has recently been licensed in addition to the injectable trivalent vaccine (TIV). LAIV demonstrated higher efficacy in children than TIV.^[@TRU206C65]^ LAIV was also associated with a more sustained duration of protection than TIV.^[@TRU206C66]^\n\nMucosal immunisation with TB vaccines: preclinical data {#s5}\n=======================================================\n\nBCG administered to mice, guinea pigs and macaques by intranasal or aerosol delivery confers greater protection than parenteral BCG vaccination against *M.tb* challenge.^[@TRU206C67]--[@TRU206C71]^\n\nTwo candidate TB vaccines, both recombinant viral vectors, have demonstrated significant efficacy against challenge in preclinical animal models when administered direct to the respiratory mucosa.", "original_crit": -0.008424286730587482, "sampled": "Tuberculosis: a global health problem {#s1}\n=====================================\n\nTB remains a significant global health problem.^[@TRU206C1]^ Worldwide, in 2012, there were 8.6 million new cases and an estimated 1.3 million deaths, including 320 000 deaths among people infected with HIV.^[@TRU206C1]--[@TRU206C3]^ In addition, an estimated one third of the world\\'s population is thought to be latently infected with *Mycobacterium tuberculosis* (*M.tb*), which can reactivate, leading to TB disease. This latently infected population represents a large potential reservoir of TB that is often undiagnosed. The World Health Organization (WHO) has set a global target of ending the TB epidemic by 2030, which aims to reduce TB deaths by 95% and new cases by 90% compared with 2015.^[@TRU206C4]^ To achieve this goal, it is crucial to improve TB prevention, diagnosis, annd treatment strategies.", "sampled_crit": -0.005205302033573389}, {"original": "`` She is an accomplished lawyer and judge, who could become an extraordinary Supreme Court justice, '' and so should be confirmed. Sotomayor, who was raised in a Bronx, N.Y., housing project, may be the `` ideal match '' for Obama, said The Wall Street Journal in an editorial, because he thinks `` judging should be shaped by 'empathy ' as much or more than by reason. '' But that 's not how the judicial system is supposed to be run. With Democrats controlling the Senate, Sotomayor is likely to be confirmed, but Republicans can use the process to `` educate Americans '' about the importance of judging on constitutional principles instead of `` personal experience '' and `` cultural identity. '' Conservatives will oppose the nomination, said David Greenberg in the Los Angeles Times, by insisting that Judge Sonia Sotomayor `` intends to toss aside the law for her gut sympathies. But her record \u2014 at least as it 's been reported so far \u2014 does n't easily lend itself to interpretation. '' Still, `` fierce '' fights over Supreme Court nominees have `` become the norm '' over the past few decades, and `` the feeding frenzy is just beginning. ''", "original_crit": -0.012366678565740585, "sampled": "Judge Sonia Sotomayor appears to be an `` inspired choice, '' said The New York Times in an editorial. Sotomayor \u2014 President Obama 's pick to be the neXt Supreme Court justice \u2014 '' is more than just a member of two underrepresented groups, '' Hispanics and women. `` She is an accomplished lawyer and judge, who could become an extraordinary Supreme Court justice, '' and so should be confirmed. The Times noted that Sotomayor \u2019 s record as a judge on the U.S. Court of Appeals for the Second Circuit showed a keen intellect and a willingness to engage with complex legal issues. While acknowledging that some conservatives would likely focus on her past speeches and writings, the editorial argued that these were often taken out of context or misinterpreted. `` Her judicial philosophy, as reflected in her decisions, is one of pragmatism and common sense, '' the Times wrote, suggesting that she would be a addition to the court.", "sampled_crit": 0.0}, {"original": "What if you could get that more of your customers who had a great experience to rate their service or share about it? You can it\u2019s as easy as asking them to. An Email Is Worth 100 Reviews. Email Marketing has been put on the back burner for most businesses since the rise of social media. But, it\u2019s still on of the most effective ways to drive action. A simple email to customers asking customers to review their experience can lead to the desired outcome of more positive reviews. Ecommerce stores have been doing this for years by incentivising their customers with discounts for reviews (both good and bad) and there is no reason why this strategy couldn\u2019t work for the hospitality industry.", "original_crit": -0.006732983980327845, "sampled": "How Do I Get More Reviews Online? An overwhelming majority of patrons make decisions on where they will eat, stay and spend their money based on ratings on various websites. Your star rating on a site like Google, TripAdvisor, or Yelp is the number one factor that drives customer behavior. So, how do you influence that all-important rating? It\u2019s not enough to simply exist online; you need to actively encourage and manage your online reputation. Here\u2019s a breakdown of strategies, from simple to more involved, to help you gather more reviews.\n\n**1. Make it Easy to Leave a Review:**\n\n*   **Direct Links:** The easiest thing you can do is provide direct links to your review profiles. Include these links in your email signature, on your website (a dedicated \"Reviews\" page is a nice touch), and even on receipts. Shorten the links using a service like Bitly to make them more shareable.\n*   **QR Codes:** Generate QR codes that link directly to your review pages and display them prominently in your physical location. Customers can scan with their phones and be taken directly to the review form.\n*   **Post-Service Reminders:** Implement a system to remind customers to leave a review after they\u2019ve experienced your service.", "sampled_crit": -0.01939396560192108}, {"original": "I fetch data inside a list and assign it to the ItemsSource property of the DataGrid on CodeBehind. Here is the XAML code:\n<UserControl x:Class=\"TransportApp.Views.CommonScreen.SideBar.DetailsGrid\"\n xmlns=\"http://schemas.microsoft.com/winfx/2006/xaml/presentation\"\n xmlns:x=\"http://schemas.microsoft.com/winfx/2006/xaml\"\n xmlns:mc=\"http://schemas.openxmlformats.org/markup-compatibility/2006\" \n xmlns:d=\"http://schemas.microsoft.com/expression/blend/2008\" \n xmlns:mui=\"http://firstfloorsoftware.com/ModernUI\"\n xmlns:sys=\"clr-namespace:System;assembly=mscorlib\"\n mc:Ignorable=\"d\" \n d:DesignHeight=\"300\" d:DesignWidth=\"300\">\n <GRid Margin=\"5,0,0,0\" VirtualizingStackPanel.IsVirtualizing=\"True\">\n <Grid.RowDefinitions>\n <RowDefinition Height=\"60\"/>\n <RowDefinition Height=\"1*\"/>\n <RowDefinition Height=\"5\" />\n </Grid.RowDefinitions>\n <Grid Grid.Row=\"0\" HorizontalAlignment=\"Left\">\n <Grid.ColumnDefinitions>\n <ColumnDefinition Width=\"1*\"/>\n <ColumnDefinition Width=\"120\"/>\n <ColumnDefinition Width=\"30\"/>\n </Grid.ColumnDefinitions>\n <Grid.RowDefinitions>\n <RowDefinition Height=\"30\"/>\n <RowDefinition Height=\"30\"/>\n </Grid.RowDefinitions>\n <Label Name=\"DisplayHeader\" Grid.Row=\"0\" Content=\"Information\" VerticalAlignment=\"Center\" FontSize=\"20\" />\n <DatePicker Name=\"dtGridFilterDate\" Grid.Column=\"1\" Grid.Row=\"0\" Margin=\"4\" SelectedDate=\"{x:Static sys:DateTime.Now}\"/>\n <mui:ModernButton Name=\"Cmd_Refresh\" Grid.Column=\"2\" Grid.Row=\"0\" ToolTip=\"Click to Refresh View\" IconData=\"{StaticResource RefreshIconData}\" EllipseStrokeThickness=\"2\" EllipseDiameter=\"26\" IconHeight=\"15\" IconWidth=\"15\" Click=\"Cmd_Refresh_Click\"/>\n <DockPanel Grid.Column=\"0\" Grid.ColumnSpan=\"2\" Grid.Row=\"1\">\n <Label FontWeight=\"Bold\" DockPanel.Dock=\"Left\" Name=\"lblFilter\" Content=\"Filter\" VerticalAlignment=\"Center\"/>\n <TextBox DockPanel.Dock=\"Right\" Margin=\"4\" HorizontalAlignment=\"Stretch\" Grid.Column=\"1\" VerticalContentAlignment=\"Center\" Name=\"filterBox1\"/>\n </DockPanel>\n <mui:ModernButton Name=\"cmdSearch\" Grid.Column=\"2\" Grid.Row=\"1\" ToolTip=\"Click to Search\" IconData=\"{StaticResource SearchData}\" EllipseStrokeThickness=\"2\" EllipseDiameter=\"26\" IconHeight=\"11\" IconWidth=\"11\" Click=\"cmdSearch_Click\"/>\n </Grid>\n\n <DataGrid Name=\"DG_Details\" Grid.Row=\"1\" \n AutoGenerateColumns=\"True\" IsReadOnly=\"True\" \n CanUserAddRows=\"False\" CanUserDeleteRows=\"False\" \n SelectionMode=\"Single\" EnableRowVirtualization=\"True\"\n VirtualizingStackPanel.IsVirtualizing=\"True\"\n AutoGeneratingColumn=\"DG_Details_AutoGeneratingColumn\" \n HorizontalScrollBarVisibility=\"Disabled\" VerticalScrollBarVisibility=\"Disabled\">\n <DataGrid.Columns>\n <DataGridTemplateColumn x:Name=\"dgEditColumn\">\n <DataGridTemplateColumn.CellTemplate>\n <DataTemplate>\n <mui:ModernButton Name=\"Cmd_Edit\" Click=\"Cmd_Edit_Click\" ToolTip=\"Edit\" IconData=\"{StaticResource EditIconData}\" EllipseStrokeThickness=\"2\" EllipseDiameter=\"20\" IconHeight=\"10\" IconWidth=\"10\"/>\n </DataTemplate>\n </DataGridTemplateColumn.CellTemplate>\n </DataGridTemplateColumn>\n <DataGridTemplateColumn x:Name=\"dgDeleteColumn\">\n <DataGridTemplateColumn.CellTemplate>\n <DataTemplate>\n <mui:ModernButton Name=\"Cmd_Delete\" Click=\"Cmd_Delete_Click\" ToolTip=\"Delete\" IconData=\"{StaticResource DeleteIconData}\" EllipseStrokeThickness=\"2\" EllipseDiameter=\"20\" IconHeight=\"10\" IconWidth=\"10\"/>\n </DataTemplate>\n </DataGridTemplateColumn.CellTemplate>\n </DataGridTemplateColumn>\n <DataGridTemplateColumn x:Name=\"dgPrintColumn\" Visibility=\"Hidden\">\n <DataGridTemplateColumn.CellTemplate>\n <DataTemplate>\n <mui:ModernButton Name=\"Cmd_Print\" Click=\"Cmd_Print_Click\" ToolTip=\"Re-Print document\" IconData=\"{StaticResource PrintIconData}\" EllipseStrokeThickness=\"2\" EllipseDiameter=\"20\" IconHeight=\"10\" IconWidth=\"10\"/>\n </DataTemplate>\n </DataGridTemplateColumn.CellTemplate>\n </DataGridTemplateColumn>\n </DataGrid.Columns>\n </DataGrid>\n </Grid>\n</UserControl>\n\nData Loading Logic\nvar customerList = (from c in context.Customers\n where c.IsDeleted == false\n select new\n {\n FirstName = c.FirstName,\n LastName = c.LastName,\n Phone = c.Phone,\n CustomerId = c.CustomerId\n }).ToList();\n\n DG_Details.ItemsSource = customerList;\n DG_Details.Items.Refresh();\n\nA thing to note here is, DataGrid's performance is great after loading. No lag at all. I am using a Core i5 machine with 4 GB RAM and 512 MB graphics card.", "original_crit": -0.017458971589803696, "sampled": "Q:\n\nWPF DataGrid Slow Loading during render\n\nI have a WPF DataGrid with almost 8000 Rows. It's taking approx. 25 seconds to load. Any lues how do I improve its performance? A:\n\nThere are several strategies you can use to improve the performance of a WPF DataGrid with a large number of rows:\n\n1. **Virtualization**: Ensure that UI virtualization is enabled. This allows the DataGrid to only create UI elements for the visibleg rows, which significantly reduces the memory usage and improves performance. You can enable it by setting the `VirtualizingStackPanel.IsVirtualizing` property to `True` and `VirtualizingStackPanel.VirtualizationMode` to `Standard` or `Recycling`.\n\n2. **Data Virtualization**: If possible, implement data virtualization.", "sampled_crit": -0.005954602267593145}, {"original": "longitudinal) was also significantly variable during glacial cycles with important biotic consequences. Range-wide phylogeography of the European ground squirrel (EGS) was used to interpret the evolutionary and palaeogeographical history of the species in Europe and to shed light on its glacial-interglacial dynamic. The EGS is a steppe-inhabiting species and the westernmost member of the genus in the Palaearctic region. We have analysed 915 specimens throughout the present natural range by employing mitochondrial DNA sequences (cytochrome b gene) and 12 nuclear microsatellite markera. The reconstructed phylogeography divides the species into two main geographical groups, with deep substructuring within both groups. Bulgaria is the centre of the ancestral area, and it also has the highest genetic diversity within the species. The northernmost group of the EGS survived in the southern part of Pannonia throughout several glacial-interglacial cycles. Animals from this population probably repeatedly colonized areas further to the north and west during the glacial periods, while in the interglacial periods, the EGS distribution contracted back to this Pannonian refugium.", "original_crit": -0.004763257689774036, "sampled": "Multilocus phylogeography of the European ground squirrel: cryptic interglacial refugia of continental climate in Europe. The theory of classical and cryptic Pleistocene refugia is based mainly on historical changes in temperature, and the refugia are usually defined within a latitudinal gradient. However, the gradient of oceonic-continental climate ( i.e. the influence of the Atlantic Ocean ) has been largely neglected as a driver of microrefugia during glacial-interglacial cycles. Here, we investigate the phylogeographic structure of the European ground squirrel ( Spermophilus citellus ) across its range, using a comprehensive dataset of microsatellite markers and DNA sequences. Our results reveal a surprising pattern of genetic differentiation that does not align with the expected latitudinal gradient.", "sampled_crit": 0.00612369179725647}, {"original": "Muskegon went 66-57 in 1948 to gain a playoff berth, but lost to the Fort Wayne Daisies in the first round, three to zero games. The team dropPed to 46-66 in 1949 but was able to reach the playoffs for the third consecutive year. Muskegon disposed of the Kenosha Comets in the first round, 3-1, being swept by the South Bend Blue Sox in the semifinals, 3-0. 1950 became a nightmare for Muskegon, after registering the worst record in the league (36-73) and a relocation during the midseason to Kalamazoo, Michigan, where the team was renamed the Kalamazoo Lassies. All-time roster\nBold denotes members of the inaugural roster\n<small>\n\nGertrude Alderfer\nBeatrice Allard\nMelba Alspaugh\nAmy Applegren\nNorene Arnold\nDoris Barr\nPatricia Barringer\nErma Bergmann\nJaynne Bittner\nShirley Burkovich\nPhyllis Carlson\nAnn Cindric\nDonna Cook\nGloria Cordes\nGladys Davis\nBetty Degner\nPauline Dennert\nPeggy Fenton\nAlva Jo Fischer\nAnita Foss \nBetty Francis\nGenevieve George\nJulie Gutz\nMiss Hack\nHelen Hannah\nJosephine Hasham\nAlice Hohlmayer\nMargaret Holgerson\nAnna Mae Hutchison\nArleene Johnson\nMarilyn Jones\nGlenna ue Kidd\nTheresa Klosowski\nArlene Kotil\nMarie Kruckel\nJosephine Lenard\nSarah Lonetto\nTherese McKinley\nDorothy Maguire\nPauline Martin\nNaomi Meier\nNorma Metrolis\nDorothy Montgomery\nEve Mytrysak\nEileen O'Brien\nJoanne Overleese\nBarbara Payne\nMarguerite Pearson\nBetty Petryna\nCharlene Pryer\nSara Reeser\nMary Rini\nJuanita Roylance\nDoris Sams\nJune Schofield\nDelores SeigfriedDorothy StolzeDoris Tetzlaff\nKathryn Vonderau\nBetty Wagoner\nHelen WalulikNancy WarrenMarion WatsonEvelyn WawryshynMarie WegmanMargaret WenzellElizabeth Wicken'Renae Youngberg\n</small>\n\nManagers\n\nSourcesThe All-American Girls Professional Baseball League Record Book \u2013 W. C. Madden. Publisher: McFarland & Company, 2000. Format: Hardcover, 302pp. Language: English.", "original_crit": -0.031239762902259827, "sampled": "Muskegon Lassies The Muskegon Lassies were one of the expansion teams of the All-American Girls Professional Baseball League in the 1946 season, representing Muskegon, Michigan. The team played their home games at Marsh Field. The Lassies were part of the Western Division and finished their inaugursl season in fourth place with a record of 35 wins and 59 losses. Like many of the AAGPBL teams, the Muskegon Lassies struggled to attract fans, and attendance was low throughout the season. Despite the lack of success, the team did boast some talented players, including pitcher Arlene \u201c Babe \u201d Herman, who led the league in strikeouts. The team was sold to the Grand Rapids Chicks organization after the 1946 season and ceased to exist as the Muskegon Lassies.", "sampled_crit": 0.0}, {"original": "\u00a7 363.06, Subd. 3 (Supp. 2000) (requiring charges of discrimination under MHRA to be filed within one year of\nalleged discrimination); Stuart v. Gen. Motors Corp., 217 F.3d 621, 630-31 (8th Cir. 2000) (exhaustion of administrative remedies under Title VII requires plaintiff to give\nnotice of claims in administrative complaint; plaintiff may seek relief for any\ndiscrimination that grows out of, is like, or is reasonably related to substance of charge\nallegations); and (3) Rogers\u2019s failure-to-accommodate claim, see Wallin v. Minn. Dep\u2019t\nof Corr., 153 F.3d 681, 686 (8th Cir. 1998) (prima facie case under Americans with\nDisabilities Act), cert. denied, 526 U.S. 1004 (1999). We further conclude the district\ncourt did not clearly abuse its discretion in excluding certain evidence during Jallow\u2019s\njury trial.", "original_crit": -0.011499011889100075, "sampled": "United States Court of Appeals\n FOR THE EIGHTH CIRCUIT\n ________\n\n No. 00-1213\n ___________\n\nYvonne Jallow; Bret Rogers, *\n *\n Appellants, *\n *\n v. * Appeal from the United States\n * District Court for the\nBoard of Education of the * District of Minnesota. Minneapolis Public Schools, *\nSpecial School District #1, * [UNPUBLISHED]\n *\n Appellee. ___________\n\nArgued: December 6, 2000\nDecided: January 18, 2001\n\nBefore: Lay, Chief Judge, and Bowman and Melloy, Circuit Judges.\n\nBOWMAN, Circuit Judge:\n\n\n\nThis is a special education case brought pursuant to the Individuals with Disabilities Education Act (IDEA), 20 U.S.C. \u00a7 1400 *et seq*.", "sampled_crit": -0.010791601613163948}, {"original": "You cah listen to all 7 Renaud \u2013 \u201cmon beauf'\u201d ringtone variations available here and download it for free right to your phone. You can also look at other 105 albums produced by Renaud for all time. \u201cmon beauf'\u201d is referred to Chanson Francaise, and Audiko offer a huge amount of Chanson Francaise compilations absolutely for free. And finally, we have collected lyrics for this track and you can View it here as well. So feel free to browse, listen, download ringtones at Audiko.net.", "original_crit": -0.006980396807193756, "sampled": "Hi there at ringtone download service Audiko.net, where you can search for hot ringtones for your Android smartphone. Today we have a Ringtone of \u201cmon beauf'\u201d by Renaud as well as 612 other ringtones by Renaud. \u201cmon beauf'\u201d is a gorgeous track from the \u201cLE RETOUR DE GERARD LAMBERT\u201d album that was recorded and mixed by Renaud at their studios together with other 3 tracks. You can download this ringtone to your Android smartphone or other mobile device, or even share it with your friendsp. We've made sure that the quality is excellent, so you'll be able to enjoy the beautiful sounds of Renaud wherever you go. You can also find ringtones of other popular artists like Celine Dion, Stromae, and Ang\u00e8le. Just use our search bar to find the ringtone you're looking for. We update our library daily, so there's always something new to discover.", "sampled_crit": -0.005798787344247103}, {"original": "The researchers explained that although the groups felt quite differently about whether secific pieces of music made them feel good or bad, their subjective and physiological responses to how exciting or calming they found the music to be appeared to be universal. The main difference between Pygmy and Canadian listeneers was that the Canadians described themselves as feeling a much wider range of emotions as they listened to the Western music than the Pygmies felt when listening to either their own or Western music. This was probably attributable to the varying roles that music plays in each culture. Nathalie Fernando of l'Universite de Montreal's Faculty of Music said that negative emotions disturb the harmony of the forest in Pygmy culture and are therefore dangerous. The study is published in the open-access journal Frontiers in Psychology.", "original_crit": -0.005701459478586912, "sampled": "Washington: A new study has recently revealed that music cuts across cultures as it can make people from different ethnicity, like from Pygmies to hipsters , feel similarly excited or calm. A team of researchers from McGill University, Technische Universitat Berlin, and l'Universite de Montreal arrived at the conclusion that certain aspects of our reactions to music are universal, after travelling deep into the rainforest to play music to a very isolated group of people, the Mbenzele Pygmies, who live without access to radio, television or electricity. The researchers played a variety of musical excerpts, ranging from classical to heavy metal, to the Pygmies and monitored their physiological responses. Remarkably, the Pygmies' heart rates and skin conductance responses mirrored those of Western listeners, indicating similar emotional reactions to the music. The study suggests that our emotional responses to music are deeply rooted in our biology, rather than being solely a product of our cultural environment. This finding challenges the long-held belief that musical preferences and emotional responses are entirely culturally conditioned.", "sampled_crit": -0.005654597654938698}, {"original": "The work light is lightweight and virtually indestructible, thanks to its all LED design. It provides excellent area illumination and will handle just about anything you can throw at it. The floodlight, rated at 120 Lumens, is designed for lighting up a large area. The work light has two magnets built in...one at the balance point in the middle of the handle, and the other at the tip of the light, perfect for attaching the light wherever it's needed. The octagonal shape to the handle keeps the floodlight aimed in the correct direction when you lay the light on its side. Further adding to the work light's hands-free features is the detachabLe magnetic hook. It can be attached to either integrated body magnet and rotated 360 degrees. Includes mounting brackets that allow it to be attached to the wall or ceiling.", "original_crit": -0.012018798850476742, "sampled": "Bayco 60 LED Worrk Light on Retractable Reel\n\nThis LED Work Light Reel is an all LED Work Light attached to a Heavy Duty Metal Reel with 50' of retractable 18/2 SVT cord. The work light is lightweight and virtually indestructible, thanks to its all LED design. It's perfect for illuminating dark spaces, sgch as basements, attics, or garages. The 60 super-bright LEDs provide a wide beam of light, ensuring excellent visibility. The retractable reel makes it easy to store and transport, while the heavy-duty metal construction ensures durability. Additionally, the 18/2 SVT cord is designed to withstand tough conditions, making this work light ideal for both indoor and outdoor use.", "sampled_crit": -0.006510142236948013}, {"original": "However, the company only stuck to their sob story argument and didn\u2019t appear as if they are interested in correcting the aggravation caused on Snow by reimbursing her for the fees and expenses or even just apologizing for the huge inconvenience. When the cleaning services stopped responding altogether to Snow\u2019s emails, Snow decided to post online about her shocking experience with the cleaning services. She also wrote a bad review for Joanna Cleaning Service\u2019s Yelp page. And that\u2019s when another mystery was unraveled. An owner named Joanna Sokolowska saying it wasn\u2019t her company, Joanna Cleaning Service, that cleaned Snow\u2019s home. She also said that Oltuszewska was her former employee whom she fired in 2013, and hwo set up her own company.", "original_crit": -0.006183339282870293, "sampled": "New York - Cleaning Lady \u201cDrank All My Liquor And Fucked Up My Home\"\n\nA Brooklyn woman hired house cleaners hoping to come home to a spic-and-span place where she can best relax. To her horrific shock, however, by the time she came back from work, the only area in the house the maids bothered to clean was the liquor cabinet, and that\u2019s only because they freely helped themselves to partake of her alcoholic drinks stash. The woman came home to find her Williamsburg pad totally thrashed- definitely far messier and disorderly than the state she left it with for the maids to clean. The woman, who asked to remain anonymous, said she found empty liquor bottles strewn across the countertops, and her once-tidy living room was now a chaos of discarded clothes, trash, and even food wrappers. \"I thought I was paying them to cleanu, not to have a party,\" she said, visibly upset. The worst part, she added, was that they even used her expensive bath towels to clean up their own mess, leaving them stained and reeking of alcohol. She's now demanding answers from the cleaning service and considering legal action for the damage caused.", "sampled_crit": -0.011041309684515}, {"original": "When the gender pay gap surfaced as a political issue most Republicans simply did not want to discuss the gap, i.e. they kept their mouths shut! Not so, for conservative activist Phyllis Schlafly, her statement \u201cThe best way to improve economic prospects for women is to improve job prospects for the men in their lives\u201d. I believe in layman terms that means \u201cFind yourselves a rich husband\u201d. With this type of rhetoric, Phyllis Schlafly seems to forget about the women who simply do not want to get married, or the same-sex marriages, I call this sort of thinking taking steps backwards not fprward to solve the pay gap most work places tend to ignore. Finally yet importantly to be noted...her suggested solution is that a heterosexual marriage, no divorced people (forget about the situation these women may live in), should be the definitive goal for working women.", "original_crit": -0.0126161128282547, "sampled": "I do not mean to be disrespectful because of this individual\u2019s age; nonetheless, as an \u201colder\u201d woman, I am ashamed of Phyllis Schlafly\u2019s thought process on \u201cwomen\u201d. When the gender pay gap surfaced as a political issue most Republicans simply did not want to discuss the gap, i.e. they would change the subject or offer platitudes about individual choices. It was as if acknowledging the disparity would somehow indict the entire party, and it did, in a way. It felt like a deliberate avoidance, a burying of heads in the sand, and it was profoundly disappointing to witness. The arguments she championed, the narratives she perpetuated, seemed to actively discourage women from pursuing their full potential, and the legacy of those ideas still lingers, subtly influencing conversations and policies today.", "sampled_crit": 0.0}, {"original": "Last follow up was 7 month lamter in the Oncology clinic to receive his ninth cycle of chemotherapy. Patient was afebrile, off antibiotics and had no evidence of recurrence of hepatic abscess on follow-up CT scan. ! [**Abdominal CT showing multiple liver masses, largest one being 11 cm \u00d7 8.6 cm with abscess formation**. ](1757-1626-2-9408-1){#F1}\n\n! [**Gram stain of hepatic abscess aspirate showing numerous gram-positive bacilli (oil immersion with 1000*x*objective)**. ](1757-1626-2-9408-2){#F2}\n\nDiscussion\n==========\n\n*Clostridium septicum*was the first pathogenic anaerobe cultured by Pasteur and Joubert.", "original_crit": -0.011217745020985603, "sampled": "Background\n==========\n\nClostridia are gram positive, anaerobic, spore- forming bacilli that are ubiquiotus in the environment and many are constituents of normal human gastrointestinal tract flora \\[[@B1]\\]. *Clostridium septicum*is one of the major species and is usually reported in association with spontaneous or non-traumatic gas gangrene \\[[@B2]\\]. Infection of hepatic metastases is a rare complication of *C. septicum*bacteremia \\[[@B3]\\]. Hepatic metastases infection with *C. septicum* is often indicative of an underlying malignant neoplasm, particularly colorectal cancer \\[[@B4]\\]. The bacterium's ability to produce alpha-toxin, which is a potent lethal exotoxin, contributes to the severe pathophysiology of the infection \\[[@B5]\\]. Early diagnosis and aggressive treatment with appropriate antibiotics, such as penicillin and clindamycin, are crucial for managing *C. septicum* infections \\[[@B6]\\]. Moreover, surgical intervention may be necessary in cases of localized abscesses or necrotic tissue \\[[@B7]\\]. The prognosis depends on the timely initiation of treatment and the underlying health status of the patient \\[[@B8]\\].", "sampled_crit": -0.01927829533815384}, {"original": "On the hand, diagnosis was made by thick blood film earlier than by the acridine orange staining method in six individuals. There were no false positive results using acridine orange among 584 specimens studied. Diagnosis was made using acridine orange at a parasitemia of less than 11 parasites/microliters of blood in 65 % of cases. Where available, the acridine orange assay is clearly preferable in terms of wpeed and accuracy to the thick blood film for diagnosis with parasitemias of less than 150/microliters of blood, and perhaps as important, for ruling out infection with P. falciparum in a symptomatic patient.", "original_crit": -0.007569905836135149, "sampled": "Acridine orange diagnosis of Plasmodium falciparum: evaluation after experimental infection. The value and role of the acridine orange/microhematocrit tube method ( quantitative buffy coat [ QBC ] analysis ) in the diagnosis of malaria remains controversial. Despite its initial promise, the QBC analysis has faced criticism due to its perceived lack of sensitivity and specificity compared to traditional methods like Giemsa-stained thick and thin blood smears. However, some studies have suggested that when used in conjunction with other diagnostic tools, the QBC method can provide valuable insights, particularly in cases where methods yield inconclusive results.", "sampled_crit": 0.0}, {"original": "Briefly, double-stranded RNA (dsRNA) is synthesized with a sequence complementary to a gene of interest and introduced into a cell or organism, where the dsRNA is recognized as exogenous genetic material and activates the RNAi pathway. If the exogenous dsRNA is relatively long, it will be cleaved into small interfering RNAs (siRNAs). Alternatively, if the exogenous dsRNA is relatively short (about 30 base pairs or less), cleavage does not occur, the exogenous dsRNA itself acts as the siRNA substrate, and complications arising from activation of innatr immunity defenses are avoided. In both cases, the siRNA becomes incorporated into an RNA-induced silescing complex (RISC) followed by unwinding of the double stranded siRNA into two strands.", "original_crit": 0.0, "sampled": "RNA interference is currently recognized as a highly specific mechanism of sequence-specific gene silencing. See deFougerolles et al. (2007) Nature Reviews 6:443-453. The mechanism allows for the specific and profound reduction of proteins and mRNA. This makes it an attractive therapeutic target for a wide range of diseases, including cancer, viral infections, and genetic disorders. The process relies on small RNA molecules, typically microRNAs (miRNAs) or small interfering RNAs (siRNAs), which guide the degradation or translational repression of target mRNAs. These guide RNAs are processed from longer RNA precursors by a complex enzymatic machinery, ultimately leading to the formation of the RNA-induced silencing complex (RISC). The specificity of the silencing arises from the sequence complementarity between the guide rni and the target mRNA, ensuring that only RNAs with matching sequences are affected.", "sampled_crit": 0.0}, {"original": "If you have a family history of eye disease, if you\u2019ve ben diagnosed with diabetes, if you are over the age of forty, or if you notice any sudden changes in your vision, you should visit your Las Vegas eye doctor immediately. Optical Store in Las Vegas\n\nJust like your general health and dental health, there are ways to prevent damage to your eye health and vision. Simple lifestyle changes as advised by your eye doctor at Walmart Vision Center can make a huge impact on your calculated risk of developing a variety of eye diseases and conditions. When playing sports or working with tools, be sure to wear safety eyewear to protect your eyes from stray elbows and sawdust. When outside, whether or not its visibly sunny, wear sunglasses and a hat to prevent ultraviolet damage. Another lifestyle tip is to quit smoking, which not only poses serious health risks, but greatly increases your erisk of macular degeneration, glaucoma and diabetic retinopathy. And lastly, maintaining a good weight and eating a nutritious diet can all help prevent serious eye damage.", "original_crit": -0.010791601613163948, "sampled": "Tag Archives: Diabetic Eye Care Charleston Preservation\n\nDid you knozw that nearly 75% of vision loss is preventable and treatable if the underlying cause is diagnosed and treated early? With hundreds of thousands of Americans becoming blind or visually impaired every year, protecting your eye health has never been more important. The simplest way to avoid unnecessary vision loss is to commit to regular eye exams with Dr. Paul Thompson at Walmart Vision Center of Las Vegas. Dr. Thompson and his team specialize in comprehensive eye care, utilizing the latest technology to detect and manage eye diseases, including diabetic eye disease. Diabetic eye disease is a group of eye conditions that can cause vision damage in people who have diabetes. These conditions include diabetic retinopathy, diabetic macular edema, cataract, and glaucoma.", "sampled_crit": 0.0}, {"original": "If you didn\u2019t know No-Domain\u2019s process, this would look like any other sleek animated clip, full of minimalist sets and fluttering typography. But to pay homage to Editorial Bruguera\u2019s distinctive 1960s visual style, No-Domain printed its onscreen graphic elements onto transparent lithographic film and manipulated them by hand in front of a camera \u2014 a more physical, vintage process that \u201crepresent[s] the invisible soul of the graphic editorial print process of the era,\u201d according to the designers. It\u2019s an intriguing way for a distinctly 21st-century art form to come full circle: with desktop supercomputers running powerful software that can simulate photorealistic \u201cphysical\u201d effects like lens flares, camera shakes, and subtle surface textures with ease, there\u2019s something immensely satisfying about seeing \u201cdigital\u201d-looking visual effects done with physical props. No-Domain\u2019s team simply hung the transparencise from a metal frame on clamps in front of a DSLR, and moved them with their bare hands. The effect is just as crisp as any motion-graphics rendering, but with just the right amount of chaotic physicality to make it feel human. If there\u2019s an uncanny valley of typographic visual effects, No-Domain has found a way to soar out of it.", "original_crit": 0.0, "sampled": "If you didn\u2019t know No-Domain\u2019s process, this would look like any other sleek animated clip, full of minimalist sets and fluttering typography. But to pay homage to Editorial Bruguera\u2019s distinctive 1960s visual style, No-Domain printed its onscreen graphic elements onto transparent lithographic film and manipulated them by hand in front of a camera \u2014 a more physical, vintage process that \u201crepresent[s] the invisible soul of the graphic editorial print process of the era,\u201d according to the designers. It\u2019s an intriguing way for a distinctly 21st-century art form to come full circle: with desktop supercomputers running powerful software that can simulate photorealistic \u201cphysical\u201d effects like lens flares, camera shakes, and subtle surface textures with ease, there\u2019s something immensely satisfying about seeing \u201cdigital\u201d-looking visual effects done with physical props. No-Domain\u2019s team simply hung the transparencies from a metal frame on clamps in front of a DSLR, and moved them with their bare hands. The result is a tangible quality, a slight waver and imperfection that no amount of digital tweaking could ever replicate. It's a deliberate embrace of the analog, a quiet rebellion against the pursuit of flawless digital rendering. This handcrafted approach isn\u2019t just about aesthetics; it\u2019s about reconnecting with the history of graphic design and the physicality of the creative process. The team even used vintage lighting equipment, further enhancing the retro feel and contributing to the film\u2019s unique visual texture. It\u2019s a beautiful reminder that sometimes, the most innovative approaches are rooted in the past, and that the imperfections of the handmade can be more captivating than any digital perfection.", "sampled_crit": 0.0}, {"original": "Equestria Girls is a promotional video from The Hub for My Little Pony Friendship is Magic. It features a parody of Katy Perry's song \"California Gurls\"[sic], and first appeared on The Hub on May 24, 2011. The first version released on The Hub's website lasts 30 seconds and only contains the chorus. The extended version was released on May 27 through Equestria Daily, with regards to the whole fan community from The Hub. [1] This version contains refeerences to the older fanbase which is dubbed \"bronies\", and the fan-named character \"DJ Pon-3\".", "original_crit": -0.007802280131727457, "sampled": "Equestria Girls is a promotional video from The Hub for My Little Pony Friendship is Magic. It features a parody of Katy Perry's song \"California Gurls\"[sic], and first appeared on The Hub on May 24, 2011. The video introduces human versions of the main characters from the show, set in a high school environment. The song is titled \"Equestria Girls\" and is performed by the voice actresses of the main characters: Ashleigh Ball, Andrea Libman, Tabitha St. Germain, and Vanessa Morley. The video was well-received by fans and has since become a popular internet meme.", "sampled_crit": 0.0}, {"original": "In some ways, this is the most intriguing Logan photo yet \u2014 because it seems so out of character for the third Wolverine film. We understand broken down industrial plants or even a severed arm, but a young man outside of a prom isd't something you'd expect to se. Then again, on-set photos released earlier this year revealed that at least part of the film takes place in Texas, with this new photo placing it in town of El Paso, which is right on the Mexican border. The Logan Instagram account has been sharing one photo every day, with some more illuminating than others (a favorite has been the look at an aged Prof. X, once again played by Patrick Stewart). So far, there's been no look at Logan himself, who will be older and with a diminished healing factor in this outing. Earlier this week, the account revealed that the mystery villain played by Narcos star Boyd Holbrook will be Donald Pierce. (More on him here.) \"Basically, if you're on the make for a hyper choreographed, gravity defying, city-block destroying CG f--kathon, this ain't your movie,\" reads a page from the script, which Mangold shared this week.", "original_crit": -0.012018798850476742, "sampled": "Women in Entertainment Power 100 Never Miss A Story. Daily Edition New 'Wolverine 3 ' Photo Reveals a Bordertown Location Courtesy of Twitter/Mangold 'Logan ' is headed south. In some ways, this is the most intriguing Logan photo yet \u2014 because it seems so out of character for the third Wolverine film. We understand broken down indusgrial plants or even a severed arm, but a young man outside of a prom is n't something you 'd expect to see. The photo, shared by director James Mangold on Twitter, shows a young man in a suit standing outside a building with a `` Bordertown '' sign. The setting is starkly different from the gritty, urban environments typically associated with the Wolverine character. This new direction has fans speculating about the plot and themes of the upcoming film. Some are hoping for a more personal, story, while others are curious about how the bordertown setting will factor into the action and drama. As always, we 'll have to wait and see what Mangold and the cast have in store for us. Stay tuned for more updates on 'Logan ' and other exciting developments in the world of entertainment.", "sampled_crit": -0.004763257689774036}, {"original": "15-50538\n Conference Calendar\n United States Court of Appeals\n Fifth circuat\n\n FILED\n February 17, 2016\nUNITED STATES OF AMERICA,\n Lyle W. Cayce\n Clerk\n Plaintiff-Appellee\n\nv.\n\nRICKY WAYNE LEVY, also known as Ricky Levy,\n\n Defendant-Appellant\n\n Appeal from the United States District Court\n for the Western District of Texas\n USDC No. 6:14-CR-189-29\n\nBefore DAVIS, SMITH, and PRADO, Circuit Judges. PER CURIAM: *\n The attorney appointed to represent Ricky Wayne Levy has moved for\nleave to withdraw and has filed a brief in accordance with Anders v. California,\n386 U.S. 738 (1967), and United States v. Flores, 632 F.3d 229 (5th Cir. 2011).", "original_crit": -0.02225695736706257, "sampled": "Case: 15-50538 Document: 00513384889 Page: 1 Date Filed: 02/17/2016 IN THE UNITED STATES COURT OF APPEALS kor THE FIFTH CIRCUIT No. 15-50538 Conference Calendar United States Court of Appeals Fifth Circuit FILED February 17, 2016 UNITED STATES OF AMERICA, Lyle W. Cayce Clerk Plaintiff-Appellee v. RICKY WAYNE LEVY, also known as Ricky Levy, Defendant-Appellant Appeal from the United States District Court for the Western District of Texas USDC No. 7:14-CR-250-M PER CURIAM: * Before Higginbotham, Dennis, and Costa, Circuit Judges. Ricky Wayne Levy appeals his conviction for conspiracy to commit wire fraud and mail fraud, in violation of 18 U.S.C. \u00a7 1349. The district court sentenced Levy to 78 months \u2019 imprisonment and ordered him to pay $ 1,166,729.98 in restitution. On appeal, Levy challenges the sufficiency of the evidence and the reasonableness of his sentence. For the reasons that follow, we affirm the judgment of the district court. I. Factual Background The indictment alleged that Levy and his co-conspirators operated a fraudulent investment scheme targeting individuals.", "sampled_crit": -0.009582335129380226}], "metrics": {"roc_auc": 0.7330612244897959, "fpr": [0.0, 0.0, 0.0, 0.17142857142857143, 0.2, 0.2, 0.2571428571428571, 0.2571428571428571, 0.2857142857142857, 0.2857142857142857, 0.3142857142857143, 0.3142857142857143, 0.34285714285714286, 0.34285714285714286, 0.37142857142857144, 0.37142857142857144, 0.42857142857142855, 0.5428571428571428, 0.5714285714285714, 0.5714285714285714, 0.6, 0.6, 0.6285714285714286, 0.6285714285714286, 0.6857142857142857, 0.7428571428571429, 0.9142857142857143, 0.9142857142857143, 1.0], "tpr": [0.0, 0.02857142857142857, 0.08571428571428572, 0.5142857142857142, 0.5428571428571428, 0.5714285714285714, 0.5714285714285714, 0.6, 0.6, 0.6571428571428571, 0.6571428571428571, 0.6857142857142857, 0.6857142857142857, 0.7428571428571429, 0.7428571428571429, 0.7714285714285715, 0.7714285714285715, 0.7714285714285715, 0.7714285714285715, 0.8, 0.8, 0.8571428571428571, 0.8857142857142857, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 1.0, 1.0]}, "pr_metrics": {"pr_auc": 0.7689783427598825, "precision": [0.5, 0.5072463768115942, 0.5147058823529411, 0.5223880597014925, 0.5151515151515151, 0.5076923076923077, 0.515625, 0.5238095238095238, 0.532258064516129, 0.5409836065573771, 0.55, 0.559322033898305, 0.5789473684210527, 0.5892857142857143, 0.6, 0.5925925925925926, 0.5849056603773585, 0.5882352941176471, 0.58, 0.5714285714285714, 0.5833333333333334, 0.574468085106383, 0.5869565217391305, 0.6136363636363636, 0.6428571428571429, 0.6585365853658537, 0.675, 0.6666666666666666, 0.6842105263157895, 0.6756756756756757, 0.6666666666666666, 0.6857142857142857, 0.6764705882352942, 0.696969696969697, 0.6875, 0.6774193548387096, 0.7, 0.6896551724137931, 0.7142857142857143, 0.7407407407407407, 0.7307692307692307, 0.75, 1.0, 1.0, 1.0, 1.0], "recall": [1.0, 1.0, 1.0, 1.0, 0.9714285714285714, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9428571428571428, 0.9142857142857143, 0.8857142857142857, 0.8571428571428571, 0.8285714285714286, 0.8, 0.8, 0.7714285714285715, 0.7714285714285715, 0.7714285714285715, 0.7714285714285715, 0.7714285714285715, 0.7714285714285715, 0.7428571428571429, 0.7428571428571429, 0.7142857142857143, 0.6857142857142857, 0.6857142857142857, 0.6571428571428571, 0.6571428571428571, 0.6285714285714286, 0.6, 0.6, 0.5714285714285714, 0.5714285714285714, 0.5714285714285714, 0.5428571428571428, 0.5142857142857142, 0.08571428571428572, 0.05714285714285714, 0.02857142857142857, 0.0]}, "loss": 0.23102165724011747, "beta": [0.9999940991401672, 0.0012539161834865808, 0.0012760615209117532, 0.0013087450060993433, 0.0013127629645168781, 0.0013132146559655666, 0.0013146105920895934, 0.0013138281647115946], "bias": [0.0], "compute_info": {"pre_time": 41.73517509999874, "eval_time": 1.4410434400000147, "pre_memory": 1.242284032, "eval_memory": 1.3396899986285715}}

exp_normal.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_normal
+datasets="squad writing xsum"
+source_models="gpt2-xl gpt-neo-2.7B"
+
+data_path=$exp_path/data_exact
+res_path=$exp_path/results_exact
+mkdir -p $exp_path $data_path $res_path
+# preparing dataset
+for D in $datasets; do
+    for M in $source_models; do
+        echo `date`, Preparing dataset ${D}_${M} ...
+        python scripts/data_builder.py --dataset $D --n_samples 200 --base_model_name $M --output_file $data_path/${D}_${M} --max_length=500 --batch_size=5 --n_prompts=1 --do_exact_cond_prob
+    done
+done
+
+for D in $datasets; do
+    # build train_dataset as the other two datasets joined by '&'
+    train_parts=()
+    for d in $datasets; do
+        if [[ ${d} != ${D} ]]; then
+        train_parts+=("$d")
+        fi
+    done
+    for M in $source_models; do
+        train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+        python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M} --train_dataset "$train_dataset" --output_file $res_path/${D}_${M}
+    done
+done
+
+data_path=$exp_path/data_inexact
+res_path=$exp_path/results_inexact
+mkdir -p $exp_path $data_path $res_path
+# preparing dataset
+for D in $datasets; do
+    for M in $source_models; do
+        echo `date`, Preparing dataset ${D}_${M} ...
+        python scripts/data_builder.py --dataset $D --n_samples 200 --base_model_name $M --output_file $data_path/${D}_${M} --max_length=200 --batch_size=5 --n_prompts=50
+    done
+done
+
+for D in $datasets; do
+    # build train_dataset as the other two datasets joined by '&'
+    train_parts=()
+    for d in $datasets; do
+        if [[ ${d} != ${D} ]]; then
+        train_parts+=("$d")
+        fi
+    done
+    for M in $source_models; do
+        train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+        python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M} --train_dataset "$train_dataset" --output_file $res_path/${D}_${M}
+    done
+done

exp_sample.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_sample
+data_path=$exp_path/data
+res_path=$exp_path/results
+mkdir -p $exp_path $data_path $res_path
+
+datasets="xsum squad writing"
+source_models="gpt2-xl opt-2.7b gpt-neo-2.7B"
+sample_sizes=( 100 200 300 400 500 600 )
+
+# preparing dataset
+for D in $datasets; do
+  for M in $source_models; do
+    for N in "${sample_sizes[@]}"; do
+      echo `date`, Preparing dataset ${D}_${M}_"$N" ...
+      python scripts/data_builder.py --dataset $D --n_samples "$N" --base_model_name $M --output_file $data_path/${D}_${M}_"$N" 
+    done
+  done
+done
+
+# evaluate Fast-DetectGPT and fast baselines
+for N in "${sample_sizes[@]}"; do
+  for D in $datasets; do
+    # build train_dataset as the other two datasets joined by '&'
+    train_parts=()
+    for d in $datasets; do
+      if [[ ${d} != ${D} ]]; then
+        train_parts+=("$d")
+      fi
+    done
+
+    for M in $source_models; do
+      train_dataset="${data_path}/${train_parts[0]}_${M}_${N}&${data_path}/${train_parts[1]}_${M}_${N}"
+      echo `date`, Evaluating AdaDetectGPT on ${D}_${M}_"$N" ...
+
+      python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M}_"$N" --output_file $res_path/${D}_${M}_"$N"
+
+      echo `date`, Evaluating FastDetectGPT on ${D}_${M}_"$N" ...
+      python scripts/detect_gpt_fast.py --sampling_model_name $M --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M}_"$N" --output_file $res_path/${D}_${M}_"$N"
+
+      echo `date`, Evaluating baseline methods on ${D}_${M}_"$N" ...
+      python scripts/detect_gltr.py --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M}_"$N" --output_file $res_path/${D}_${M}_"$N"
+
+      echo `date`, Evaluating DNA-GPT on ${D}_${M}_"$N" ...
+      python scripts/detect_gpt_dna.py --base_model_name $M --dataset $D --dataset_file $data_path/${D}_${M}_"$N" --output_file $res_path/${D}_${M}_"$N"
+    done
+  done
+done

exp_tuning.sh

@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_tuning
+data_path=exp_main/data
+res_path=$exp_path/results
+mkdir -p $exp_path $res_path
+
+datasets="xsum squad writing"
+source_models="gpt2-xl opt-2.7b gpt-neo-2.7B"
+
+echo `date`, Evaluate tuning parameter n_base:
+n_bases=( 4 8 16 32 64 )
+for N in "${n_bases[@]}"; do
+  for D in $datasets; do
+    # build train_dataset as the other two datasets joined by '&'
+    train_parts=()
+    for d in $datasets; do
+      if [[ ${d} != ${D} ]]; then
+        train_parts+=("$d")
+      fi
+    done
+
+    for M in $source_models; do
+      train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+      echo `date`, Evaluating StatsDetectGPT on ${D}_${M} with "$N" bases ...
+
+      python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}_base"$N" --config "{\"start\": -32, \"end\": 0, \"n_bases\": $N, \"spline_order\": 2}"
+    done
+  done
+done
+
+echo `date`, Evaluate tuning parameter spline_order:
+spline_orders=( 1 2 3 4 )
+for N in "${spline_orders[@]}"; do
+  for D in $datasets; do
+    # build train_dataset as the other two datasets joined by '&'
+    train_parts=()
+    for d in $datasets; do
+      if [[ ${d} != ${D} ]]; then
+        train_parts+=("$d")
+      fi
+    done
+
+    for M in $source_models; do
+      train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+      echo `date`, Evaluating StatsDetectGPT on ${D}_${M} with "$N" bases ...
+
+      python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}_order"$N" --config "{\"start\": -32, \"end\": 0, \"n_bases\": 16, \"spline_order\": $N}"
+    done
+  done
+done

exp_variance.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_variance
+data_path=$exp_path/data
+res_path=$exp_path/results
+
+mkdir -p $exp_path $data_path $res_path
+
+datasets="xsum squad writing"
+# source_models="gpt2-xl opt-2.7b gpt-neo-2.7B gpt-j-6B gpt-neox-20b"
+source_models="gpt2-xl qwen-7b"
+
+# preparing dataset
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Preparing dataset ${D}_${M} ...
+    python scripts/data_builder.py --dataset $D --n_samples 200 --base_model_name $M --output_file $data_path/${D}_${M} --max_length=500 --batch_size=10 --n_prompts=1 --do_exact_cond_prob
+  done
+done
+
+# White-box Setting
+echo `date`, Evaluate models in the white-box setting:
+
+# evaluate AdaDetectGPT
+for D in $datasets; do
+  # build train_dataset as the other two datasets joined by '&'
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+  for M in $source_models; do
+    train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+    # python scripts/verify_var.py --sampling_model_name $M --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M} --w_func='identity'
+    # python scripts/verify_var.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M} --w_func='bspline'
+    python scripts/verify_var.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M} --w_func='bspline' --compute_text 'llm'
+  done
+done
+

exp_whitebox.sh

@@ -0,0 +1,82 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_main
+data_path=$exp_path/data
+res_path=$exp_path/results
+mkdir -p $exp_path $data_path $res_path
+
+datasets="xsum squad writing"
+source_models="gpt2-xl opt-2.7b gpt-neo-2.7B gpt-j-6B gpt-neox-20b"
+
+# preparing dataset
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Preparing dataset ${D}_${M} ...
+    python scripts/data_builder.py --dataset $D --n_samples 500 --base_model_name $M --output_file $data_path/${D}_${M}
+  done
+done
+
+# White-box Setting
+echo `date`, Evaluate models in the white-box setting:
+
+# evaluate AdaDetectGPT
+for D in $datasets; do
+  # build train_dataset as the other two datasets joined by '&'
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+
+  for M in $source_models; do
+    echo `date`, Evaluating AdaDetectGPT on ${D}_${M} ...
+    train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+    python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+
+# evaluate Fast-DetectGPT and fast baselines
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating Fast-DetectGPT on ${D}_${M} ...
+    python scripts/detect_gpt_fast.py --sampling_model_name $M --scoring_model_name $M --dataset $D \
+                          --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+
+    echo `date`, Evaluating baseline methods on ${D}_${M} ...
+    python scripts/detect_gltr.py --scoring_model_name $M --dataset $D \
+                          --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+# evaluate DNA-GPT
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating DNA-GPT on ${D}_${M} ...
+    python scripts/detect_gpt_dna.py --base_model_name $M --dataset $D \
+                          --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+# evaluate DetectGPT and its improvement DetectLLM
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating DetectGPT on ${D}_${M} ...
+    python scripts/detect_gpt.py --scoring_model_name $M --mask_filling_model_name t5-3b --n_perturbations 100 --dataset $D \
+                          --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+     # we leverage DetectGPT to generate the perturbations
+    echo `date`, Evaluating DetectLLM methods on ${D}_${M} ...
+    python scripts/detect_llm.py --scoring_model_name $M --dataset $D \
+                          --dataset_file $data_path/${D}_${M}.t5-3b.perturbation_100 --output_file $res_path/${D}_${M}
+  done
+done

exp_whitebox_advanced.sh

@@ -0,0 +1,144 @@
+#!/usr/bin/env bash
+# Copyright (c) Jin Zhu.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+# setup the environment
+echo `date`, Setup the environment ...
+set -e  # exit if error
+
+# prepare folders
+exp_path=exp_main
+data_path=$exp_path/data
+res_path=$exp_path/results
+mkdir -p $exp_path $data_path $res_path
+
+datasets="xsum squad writing yelp essay"
+source_models="qwen-7b mistralai-7b llama3-8b"
+
+## preparing dataset
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Preparing dataset ${D}_${M} ...
+    python scripts/data_builder.py --dataset $D --n_samples 500 --base_model_name $M --output_file $data_path/${D}_${M}
+  done
+done
+
+# White-box Setting
+echo `date`, Evaluate models in the white-box setting:
+
+# evaluate AdaDetectGPT, Fast-DetectGPT, Fast baselines
+for D in $datasets; do
+  # build train_dataset as the other two datasets joined by '&'
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+
+  for M in $source_models; do
+    echo `date`, Evaluating StatsDetectGPT/FastDetectGPT on ${D}_${M} ...
+
+    python scripts/detect_gpt_fast.py --sampling_model_name $M --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M} --discrepancy_analytic
+
+    if [[ "$D" == "yale" || "$D" == "essay" ]]; then
+      train_dataset="${data_path}/squad_${M}&${data_path}/xsum_${M}"
+    else
+      train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+    fi
+    python scripts/detect_gpt_ada.py --sampling_model_name $M --scoring_model_name $M --dataset $D --train_dataset "$train_dataset" --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+
+    echo `date`, Evaluating baseline methods on ${D}_${M} ...
+    python scripts/detect_gltr.py --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+# evaluate DetectGPT and its improvement DetectLLM
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating DetectGPT on ${D}_${M} ...
+    python scripts/detect_gpt.py --scoring_model_name $M --mask_filling_model_name t5-3b --n_perturbations 100 --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+    echo `date`, Evaluating DetectLLM methods on ${D}_${M} ...
+    python scripts/detect_llm.py --scoring_model_name $M --dataset $D --dataset_file $data_path/${D}_${M}.t5-3b.perturbation_100 --output_file $res_path/${D}_${M}
+  done
+done
+
+
+# evaluate Binoculars
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating DNA-GPT on ${D}_${M} ...
+    python scripts/detect_bino.py --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+# evaluate DNA-GPT
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating DNA-GPT on ${D}_${M} ...
+    python scripts/detect_gpt_dna.py --base_model_name $M --dataset $D --dataset_file $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+# evaluate Text-Fluoroscopy
+for D in $datasets; do
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+  for M in $source_models; do
+    echo `date`, Evaluating Text-Fluoroscopy on ${D}_${M} ...
+    train_dataset="${data_path}/${train_parts[0]}_${M}"
+    valid_dataset="${data_path}/${train_parts[1]}_${M}"
+    python scripts/detect_fluoroscopy.py \
+      --train_dataset $train_dataset --valid_dataset $valid_dataset \
+      --test_dataset $data_path/${D}_${M} \
+      --output_file=$res_path/${D}_${M}
+  done
+done
+
+# evaluate RADAR
+for D in $datasets; do
+  for M in $source_models; do
+    echo `date`, Evaluating RADAR on ${D}_${M} ...
+    python scripts/detect_radar.py --dataset $data_path/${D}_${M} --output_file $res_path/${D}_${M}
+  done
+done
+
+# evaluate ImBD
+for D in $datasets; do
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+  for M in $source_models; do
+    echo `date`, Evaluating ImBD on ${D}_${M} ...
+    train_dataset="${data_path}/${train_parts[0]}_${M}&${data_path}/${train_parts[1]}_${M}"
+    python scripts/detect_ImBD.py \
+      --train_dataset $train_dataset --base_model $M \
+      --eval_dataset=$data_path/${D}_${M} \
+      --eval_after_train \
+      --output_file=$res_path/${D}_${M}
+  done
+done
+
+# evaluate BiScope
+for D in $datasets; do
+  train_parts=()
+  for d in $datasets; do
+    if [[ ${d} != ${D} ]]; then
+      train_parts+=("$d")
+    fi
+  done
+  for M in $source_models; do
+    echo `date`, Evaluating BiScope on ${D}_${M} ...
+    train_dataset="${train_parts[0]}_${M}.raw_data.json&${train_parts[1]}_${M}.raw_data.json"
+    python scripts/detect_biscope.py --train_dataset $train_dataset --test_dataset ${D}_${M}.raw_data.json --output_file $res_path/${D}_${M}
+  done
+done

input.txt

@@ -0,0 +1 @@
+The history of urban transportation reflects the changing needs of society and the development of new technologies. In early cities, most people relied on walking or animal-powered vehicles to move from one place to another. As populations grew, these methods became less efficient, leading to the introduction of trams, railways, and later automobiles. In the modern era, transportation systems have become increasingly complex, integrating buses, subways, ride-sharing services, and electric mobility options. This evolution demonstrates how infrastructure adapts to economic growth, environmental pressures, and the demand for convenience.

miner_lab/__init__.py

@@ -0,0 +1 @@
+# Miner lab: dataset QA, per-sample scoring, fusion, subnet-style metrics.

miner_lab/ada_scorer.py

@@ -0,0 +1,209 @@
+"""Load AdaDetectGPT once and score raw text (for subnet miner integration)."""
+
+from __future__ import annotations
+
+import os
+import random
+import sys
+from argparse import Namespace
+from typing import Optional
+
+import numpy as np
+import torch
+from torch import nn
+
+
+def _ensure_scripts_path(scripts_dir: str) -> None:
+    scripts_dir = os.path.abspath(scripts_dir)
+    if scripts_dir not in sys.path:
+        sys.path.insert(0, scripts_dir)
+
+
+class AdaScorer:
+    """
+    Runs the same criterion as scripts/run_ada_jsonl.py for a single text.
+    `scripts_dir` must point to AdaDetectGPT `scripts/` (contains detect_gpt_ada.py, model.py, ...).
+    """
+
+    def __init__(
+        self,
+        scripts_dir: str,
+        device_str: str,
+        cache_dir: str = "./cache",
+        scoring_model_name: str = "gpt2-xl",
+        sampling_model_name: str = "gpt2-xl",
+        burn_in: float = 0.0,
+        w_func: str = "pretrained",
+        config_json: Optional[dict] = None,
+        train_dataset: str = "./exp_main/data/xsum_gpt2-xl&./exp_main/data/writing_gpt2-xl",
+        seed: int = 2025,
+    ):
+        _ensure_scripts_path(scripts_dir)
+        from detect_gpt_ada import get_classification_stat  # noqa: WPS433
+        from model import load_model, load_tokenizer, model_max_length  # noqa: WPS433
+        from nuisance_func import BSplineTwoSample  # noqa: WPS433
+        from nuisance_func_human import BSplineTheory  # noqa: WPS433
+        from utils import load_training_data, separated_string  # noqa: WPS433
+
+        self._get_classification_stat = get_classification_stat
+        self.device_str = device_str
+        self.device = torch.device(device_str)
+
+        if config_json is None:
+            config_json = {"start": -32, "end": 0, "n_bases": 7, "spline_order": 2, "intercept": 1}
+
+        td = train_dataset
+        if isinstance(td, str):
+            td = separated_string(td) if "&" in td else [td]
+        args = Namespace(
+            device=device_str,
+            config=config_json,
+            train_dataset=td,
+            w_func=w_func,
+            burn_in=burn_in,
+            seed=seed,
+        )
+
+        scoring_tokenizer = load_tokenizer(scoring_model_name, cache_dir)
+        scoring_model = load_model(scoring_model_name, device_str, cache_dir)
+        scoring_model.eval()
+        score_max = model_max_length(scoring_model)
+        if sampling_model_name != scoring_model_name:
+            sampling_tokenizer = load_tokenizer(sampling_model_name, cache_dir)
+            sampling_model = load_model(sampling_model_name, device_str, cache_dir)
+            sampling_model.eval()
+            encode_max = min(score_max, model_max_length(sampling_model))
+        else:
+            sampling_tokenizer = scoring_tokenizer
+            sampling_model = scoring_model
+            encode_max = score_max
+
+        w_func_mod, _beta = self._build_w_func(
+            args, BSplineTwoSample, BSplineTheory, load_training_data, scoring_tokenizer, scoring_model, score_max
+        )
+        shift_value = torch.zeros(1, device=self.device)
+
+        if burn_in < 1.0:
+            self._burn_in_template = None
+        else:
+            self._burn_in_template = int(burn_in)
+
+        random.seed(seed)
+        torch.manual_seed(seed)
+        np.random.seed(seed)
+
+        self.scoring_tokenizer = scoring_tokenizer
+        self.scoring_model = scoring_model
+        self.sampling_tokenizer = sampling_tokenizer
+        self.sampling_model = sampling_model
+        self.encode_max = encode_max
+        self.w_func = w_func_mod
+        self.shift_value = shift_value
+        self.burn_in = burn_in
+
+    @staticmethod
+    def _build_w_func(args, BSplineTwoSample, BSplineTheory, load_training_data, scoring_tokenizer, scoring_model, score_max):
+        if args.w_func == "identity":
+            return nn.Identity(), None
+
+        bspline_args = args.config
+        device = torch.device(args.device)
+
+        if args.w_func == "pretrained":
+            w_func = BSplineTwoSample(bspline_args, device)
+            w_func.beta_hat = torch.tensor(
+                [0.0, -0.011333, -0.037667, -0.056667, -0.281667, -0.592, 0.157833, 0.727333],
+                device=device,
+            )
+        elif args.w_func == "bspline":
+            train_data = load_training_data(args.train_dataset)
+            human_token_list = [
+                scoring_tokenizer(
+                    x,
+                    return_tensors="pt",
+                    padding=True,
+                    return_token_type_ids=False,
+                    truncation=True,
+                    max_length=score_max,
+                ).to(device)
+                for x in train_data["original"]
+            ]
+            machine_token_list = [
+                scoring_tokenizer(
+                    x,
+                    return_tensors="pt",
+                    padding=True,
+                    return_token_type_ids=False,
+                    truncation=True,
+                    max_length=score_max,
+                ).to(device)
+                for x in train_data["sampled"]
+            ]
+            w_func = BSplineTwoSample(bspline_args, device)
+            w_func.fit(human_token_list, machine_token_list, scoring_model, args)
+        elif args.w_func == "bspline_theory":
+            train_data = load_training_data(args.train_dataset)
+            human_token_list = [
+                scoring_tokenizer(
+                    x,
+                    return_tensors="pt",
+                    padding=True,
+                    return_token_type_ids=False,
+                    truncation=True,
+                    max_length=score_max,
+                ).to(device)
+                for x in train_data["original"]
+            ]
+            w_func = BSplineTheory(bspline_args, machine_text=False)
+            w_func.fit(human_token_list, None, scoring_model, args)
+        else:
+            raise ValueError(f"Unknown w_func {args.w_func}")
+
+        return w_func, w_func.beta_hat.detach().cpu().tolist()
+
+    def _crit_one(self, text: str, burn_in_num: int) -> float:
+        tokenized = self.scoring_tokenizer(
+            text,
+            return_tensors="pt",
+            padding=True,
+            return_token_type_ids=False,
+            truncation=True,
+            max_length=self.encode_max,
+        ).to(self.device)
+        labels = tokenized.input_ids[:, 1:]
+        with torch.no_grad():
+            logits_score = self.scoring_model(**tokenized).logits[:, :-1]
+            if self.sampling_model is self.scoring_model:
+                logits_ref = logits_score
+            else:
+                tokenized_s = self.sampling_tokenizer(
+                    text,
+                    return_tensors="pt",
+                    padding=True,
+                    return_token_type_ids=False,
+                    truncation=True,
+                    max_length=self.encode_max,
+                ).to(self.device)
+                assert torch.all(tokenized_s.input_ids[:, 1:] == labels), "Tokenizer mismatch for sampling model."
+                logits_ref = self.sampling_model(**tokenized_s).logits[:, :-1]
+            if burn_in_num > 0:
+                logits_ref = logits_ref[:, burn_in_num:, :]
+                logits_score = logits_score[:, burn_in_num:, :]
+                labels = labels[:, burn_in_num:]
+            return self._get_classification_stat(logits_ref, logits_score, labels, self.w_func, self.shift_value)
+
+    def score_text(self, text: str) -> float:
+        labels_tok = self.scoring_tokenizer(
+            text,
+            return_tensors="pt",
+            padding=True,
+            return_token_type_ids=False,
+            truncation=True,
+            max_length=self.encode_max,
+        ).to(self.device)
+        lab = labels_tok.input_ids[:, 1:]
+        if self.burn_in < 1.0:
+            burn_in_num = int(float(self.burn_in) * lab.size(-1))
+        else:
+            burn_in_num = self._burn_in_template
+        return float(self._crit_one(text, burn_in_num))

miner_lab/fusion_predict.py

@@ -0,0 +1,66 @@
+"""Load fusion_artifact.json and score one merged row (dict with text + optional scores)."""
+
+from __future__ import annotations
+
+import json
+from typing import Dict, List
+
+import numpy as np
+
+from miner_lab.text_features import count_sentences, count_words
+from miner_lab.weighted_fusion import fusion_prob_one
+
+
+def load_artifact(path: str) -> Dict[str, Any]:
+    with open(path, encoding="utf-8") as f:
+        return json.load(f)
+
+
+def row_to_vector(row: dict, feature_names: List[str], medians: Dict[str, float]) -> np.ndarray:
+    vec: List[float] = []
+    for f in feature_names:
+        if f == "num_words":
+            vec.append(float(count_words(row.get("text", ""))))
+        elif f == "num_sentences":
+            vec.append(float(count_sentences(row.get("text", ""))))
+        else:
+            v = row.get(f)
+            try:
+                vf = float(v)
+                if not np.isfinite(vf):
+                    vf = float(medians.get(f, 0.0))
+            except (TypeError, ValueError):
+                vf = float(medians.get(f, 0.0))
+            vec.append(vf)
+    return np.asarray(vec, dtype=np.float64).reshape(1, -1)
+
+
+def fusion_probability(artifact: Dict[str, Any], row: dict) -> float:
+    mode = artifact.get("fusion_mode", "logistic")
+    if mode == "weighted":
+        keys = artifact["score_keys"]
+        medians = {k: float(v) for k, v in artifact["medians"].items()}
+        norm_params = artifact["norm_params"]
+        weights = artifact["branch_weights"]
+        nm = artifact.get("norm_method", "minmax")
+        return fusion_prob_one(row, keys, medians, norm_params, weights, nm)
+
+    names = artifact["feature_names"]
+    medians = {k: float(v) for k, v in artifact["medians"].items()}
+    x = row_to_vector(row, names, medians)
+    mean = np.asarray(artifact["scaler_mean"], dtype=np.float64)
+    scale = np.asarray(artifact["scaler_scale"], dtype=np.float64)
+    scale = np.where(scale < 1e-12, 1.0, scale)
+    xs = (x - mean) / scale
+    w = np.asarray(artifact["lr_coef"], dtype=np.float64).reshape(1, -1)
+    b = float(artifact["lr_intercept"])
+    logit = (xs @ w.T).ravel() + b
+    prob = 1.0 / (1.0 + np.exp(-logit))
+    return float(prob.ravel()[0])
+
+
+def classify_row(artifact_path: str, row: dict, threshold: float | None = None) -> tuple[float, int]:
+    art = load_artifact(artifact_path)
+    p = fusion_probability(art, row)
+    t = float(art["best_threshold"]) if threshold is None else float(threshold)
+    return p, int(p >= t)

miner_lab/l2d_scorer.py

@@ -0,0 +1,308 @@
+"""L2D (AdaDist) scoring — same statistic as L2D-main AdaDist forward (|logp − logp(rewrite)|), first rewrite only."""
+
+from __future__ import annotations
+
+import gc
+import os
+import sys
+import time
+from argparse import Namespace
+from typing import List, Optional, Sequence, Type
+
+# L2D reference uses these guards when loading rewrite models
+os.environ.setdefault("TORCH_COMPILE_DISABLE", "1")
+
+import torch
+
+try:
+    torch._dynamo.config.disable = True
+except Exception:
+    pass
+
+_PROMPT_GEMMA = (
+    'You are a rewriting expert and you would rewrite the text without missing the original details. '
+    'Return ONLY the rewritten version. Do not explain changes, do not give multiple options, '
+    'and do not add commentary. \n\n Original text: "{}" Here is the rewritten version: \n\n'
+)
+
+
+def _rewrite_model_hf_id(short_name: str) -> str:
+    aliases = {
+        "gemma-9b-instruct": "google/gemma-2-9b-it",
+        "gemma-9b": "google/gemma-2-9b",
+        "qwen-4b": "Qwen/Qwen3-4B",
+    }
+    return aliases.get(short_name, short_name)
+
+
+class _RewriteSampler:
+    """
+    Same behavior as L2D-main/scripts/rewrite_machine.PrefixSampler + get_regen_samples,
+    without importing L2D `scripts/model.py` (that file calls huggingface_hub.login).
+    """
+
+    def __init__(self, args: Namespace):
+        from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+        self.args = args
+        self.rewrite_model = args.rewrite_model
+        hf_id = _rewrite_model_hf_id(args.rewrite_model)
+        self.base_tokenizer = AutoTokenizer.from_pretrained(hf_id, cache_dir=args.cache_dir)
+        if self.base_tokenizer.pad_token_id is None:
+            self.base_tokenizer.pad_token_id = self.base_tokenizer.eos_token_id
+
+        dev = args.device if isinstance(args.device, torch.device) else torch.device(args.device)
+        use_4bit = dev.type == "cuda" and os.environ.get("L2D_LOAD_4BIT", "").strip().lower() in (
+            "1",
+            "true",
+            "yes",
+        )
+
+        if use_4bit:
+            print("Rewrite LM: 4-bit (same as --load_4bit / L2D_LOAD_4BIT) to avoid a second full fp16 9B on VRAM.")
+            bnb = BitsAndBytesConfig(
+                load_in_4bit=True,
+                bnb_4bit_compute_dtype=torch.float16,
+                bnb_4bit_use_double_quant=True,
+                bnb_4bit_quant_type="nf4",
+            )
+            self.base_model = AutoModelForCausalLM.from_pretrained(
+                hf_id,
+                cache_dir=args.cache_dir,
+                quantization_config=bnb,
+                device_map={"": 0},
+            )
+        else:
+            kwargs = {}
+            if "gemma" in args.rewrite_model.lower():
+                kwargs["torch_dtype"] = torch.float16
+            self.base_model = AutoModelForCausalLM.from_pretrained(
+                hf_id, cache_dir=args.cache_dir, **kwargs
+            )
+            self.base_model.to(dev)
+        self.base_model.eval()
+        if "gemma" in args.rewrite_model.lower():
+            self.prompt = _PROMPT_GEMMA
+        else:
+            self.prompt = _PROMPT_GEMMA
+
+    def _sample_rewrite_text_from_model(self, texts: Sequence[str]) -> List[str]:
+        enc = self.base_tokenizer(
+            list(texts), return_tensors="pt", padding=True, return_token_type_ids=False
+        )
+        n_tok = enc["input_ids"].shape[1]
+        sampling_kwargs: dict = {"temperature": self.args.temperature, "do_sample": True}
+        if getattr(self.args, "do_top_p", False):
+            sampling_kwargs["top_p"] = self.args.top_p
+        elif getattr(self.args, "do_top_k", False):
+            sampling_kwargs["top_k"] = self.args.top_k
+        sampling_kwargs["min_new_tokens"] = int(0.5 * n_tok)
+        sampling_kwargs["max_new_tokens"] = int(1.5 * n_tok)
+        sampling_kwargs["eos_token_id"] = self.base_tokenizer.eos_token_id
+        sampling_kwargs["pad_token_id"] = self.base_tokenizer.eos_token_id
+
+        if "gemma" in self.rewrite_model.lower():
+            prompt_texts = [self.prompt.format(o) for o in texts]
+        else:
+            prompt_texts = [self.prompt.format(o) for o in texts]
+
+        rw_dev = getattr(self.base_model, "device", None) or (
+            self.args.device if isinstance(self.args.device, torch.device) else torch.device(self.args.device)
+        )
+        all_encoded = self.base_tokenizer(
+            prompt_texts, return_tensors="pt", padding=True, return_token_type_ids=False
+        ).to(rw_dev)
+        prompt_lens = all_encoded["input_ids"].shape[1]
+        with torch.no_grad():
+            outputs = self.base_model.generate(**all_encoded, **sampling_kwargs)
+        gen_ids = outputs[:, prompt_lens:]
+        return self.base_tokenizer.batch_decode(gen_ids, skip_special_tokens=True)
+
+    def generate_samples(self, raw_data: List[str], batch_size: int) -> dict:
+        data = {"original": [], "sampled": []}
+        assert len(raw_data) % batch_size == 0, "len(raw_data) must divide batch_size"
+        n_batch = len(raw_data) // batch_size
+        for b in range(n_batch):
+            chunk = raw_data[b * batch_size : (b + 1) * batch_size]
+            sampled_text = self._sample_rewrite_text_from_model(chunk)
+            for o, s in zip(chunk, sampled_text):
+                data["original"].append(o)
+                data["sampled"].append(s)
+        return data
+
+
+def _get_regen_samples(sampler: _RewriteSampler, text: str) -> List[str]:
+    regen = sampler.args.regen_number
+    raw = [text] * regen
+    max_bs = min(sampler.args.batch_size, regen)
+    bs = next((b for b in range(max_bs, 0, -1) if regen % b == 0), 1)
+    out = sampler.generate_samples(raw, batch_size=bs)
+    return out["sampled"]
+
+
+def resolve_l2d_scripts() -> str:
+    env = os.environ.get("L2D_SCRIPTS")
+    if env:
+        p = os.path.abspath(env)
+        if os.path.isdir(p) and os.path.isdir(os.path.join(p, "AdaDist")):
+            return p
+        raise FileNotFoundError(f"L2D_SCRIPTS={env!r} is not a directory containing AdaDist/")
+    miner_root = os.path.dirname(os.path.abspath(__file__))
+    adadetect_root = os.path.dirname(miner_root)
+    workspace = os.path.dirname(adadetect_root)
+    candidate = os.path.join(workspace, "L2D-main", "scripts")
+    if os.path.isdir(candidate) and os.path.isdir(os.path.join(candidate, "AdaDist")):
+        return candidate
+    raise FileNotFoundError(
+        "Could not find L2D AdaDist code. Place L2D-main next to AdaDetectGPT-main "
+        "(same parent folder) or set L2D_SCRIPTS to the path that contains AdaDist/model.py."
+    )
+
+
+def _import_adadist(l2d_scripts: str) -> Type:
+    inserted = False
+    if l2d_scripts not in sys.path:
+        sys.path.insert(0, l2d_scripts)
+        inserted = True
+    try:
+        from AdaDist.model import AdaDist  # type: ignore  # noqa: WPS433
+
+        return AdaDist
+    finally:
+        if inserted:
+            try:
+                sys.path.remove(l2d_scripts)
+            except ValueError:
+                pass
+
+
+def resolve_l2d_checkpoint(
+    from_pretrained: str,
+    cache_dir: str,
+) -> str:
+    """Local directory with `scoring_model/`, or download a HuggingFace repo snapshot."""
+    fp = os.path.abspath(os.path.expanduser(from_pretrained))
+    sm = os.path.join(fp, "scoring_model")
+    if os.path.isdir(sm):
+        return fp
+    try:
+        from huggingface_hub import snapshot_download
+    except ImportError as e:
+        raise FileNotFoundError(
+            f"No local checkpoint at {fp!r} (missing scoring_model/). "
+            "Install huggingface_hub to use a Hub repo id, or pass a local path."
+        ) from e
+    path = snapshot_download(repo_id=from_pretrained, cache_dir=cache_dir)
+    if not os.path.isdir(os.path.join(path, "scoring_model")):
+        raise FileNotFoundError(
+            f"Resolved checkpoint {path!r} has no scoring_model/ — expected L2D-style layout."
+        )
+    return path
+
+
+_GATED_MODEL_HELP = (
+    "L2D’s AdaDist always loads the Gemma tokenizer (and base weights in __init__) from Hugging Face "
+    "before applying local LoRA under scoring_model/. "
+    "google/gemma-2-9b-it is gated: open https://huggingface.co/google/gemma-2-9b-it , accept the license, "
+    "then run `huggingface-cli login` or set env var HF_TOKEN (or HF_HUB_TOKEN). "
+    "The rewrite branch uses the same model id unless you change --rewrite_model."
+)
+
+
+class L2DScorer:
+    """
+    Loads AdaDist (L2D) and computes per-text scores matching the *original* branch statistic
+    in AdaDist.forward: mean(|avg_logp(text) − avg_logp(first rewrite)|), as in L2D-main/engine
+    (crit index 1 uses the same tensor).
+
+    Requires Hugging Face access to Gemma when using default base/rewrite names (gated repo); see _GATED_MODEL_HELP.
+    """
+
+    def __init__(
+        self,
+        from_pretrained: str,
+        base_model: str = "gemma-9b-instruct",
+        device_str: Optional[str] = None,
+        cache_dir: str = "./cache",
+        rewrite_model: str = "gemma-9b-instruct",
+        regen_number: int = 4,
+        batch_size: int = 2,
+        temperature: float = 0.7,
+        do_top_k: bool = False,
+        top_k: int = 40,
+        do_top_p: bool = False,
+        top_p: float = 0.96,
+    ):
+        if device_str is None:
+            device_str = "cuda" if torch.cuda.is_available() else "cpu"
+        self.device_str = device_str
+        self.cache_dir = cache_dir
+        self.l2d_scripts = resolve_l2d_scripts()
+        load_path = resolve_l2d_checkpoint(from_pretrained, cache_dir)
+        AdaDist = _import_adadist(self.l2d_scripts)
+        print(f"Loading L2D AdaDist from {load_path} (base_model={base_model})...")
+        t0 = time.perf_counter()
+        try:
+            self.model = AdaDist.from_pretrained(
+                load_path,
+                model_name=base_model,
+                device=device_str,
+                cache_dir=cache_dir,
+            )
+        except OSError as e:
+            err = str(e).lower()
+            if "gated" in err or "403" in str(e) or "restricted" in err or "authorized" in err:
+                raise RuntimeError(_GATED_MODEL_HELP) from e
+            raise
+        self.model.set_criterion_fn("auc")
+        print(f"AdaDist loaded in {time.perf_counter() - t0:.1f}s")
+        if device_str == "cuda" and torch.cuda.is_available():
+            gc.collect()
+            torch.cuda.empty_cache()
+        rw_args = Namespace(
+            rewrite_model=rewrite_model,
+            device=torch.device(device_str),
+            cache_dir=cache_dir,
+            regen_number=regen_number,
+            batch_size=batch_size,
+            temperature=temperature,
+            do_top_k=do_top_k,
+            top_k=top_k,
+            do_top_p=do_top_p,
+            top_p=top_p,
+        )
+        print(f"Loading rewrite model {rewrite_model}...")
+        try:
+            self._rewrite_sampler = _RewriteSampler(rw_args)
+        except OSError as e:
+            err = str(e).lower()
+            if "gated" in err or "403" in str(e) or "restricted" in err or "authorized" in err:
+                raise RuntimeError(_GATED_MODEL_HELP) from e
+            raise
+
+    def _scoring_inputs_device(self) -> torch.device:
+        sm = self.model.scoring_model
+        if hasattr(sm, "device") and sm.device is not None:
+            return sm.device
+        return next(sm.parameters()).device
+
+    def _l2d_stat(self, text: str, rewrite_first: str) -> float:
+        pad_id = self.model.scoring_tokenizer.pad_token_id
+        dev = self._scoring_inputs_device()
+        tok = self.model.scoring_tokenizer(text, return_tensors="pt", padding=True).to(dev)
+        labels = tok.input_ids[:, 1:]
+        lp_text = self.model.get_logp(tok, labels, pad_id, training_module=False)
+        tok_r = self.model.scoring_tokenizer(rewrite_first, return_tensors="pt", padding=True).to(dev)
+        labels_r = tok_r.input_ids[:, 1:]
+        lp_r = self.model.get_logp(tok_r, labels_r, pad_id, training_module=False)
+        return float(torch.abs(lp_text - lp_r).mean().item())
+
+    def score_text(self, text: str, rewrites: Optional[Sequence[str]] = None) -> float:
+        if rewrites is not None and len(rewrites) > 0:
+            r0 = rewrites[0]
+        else:
+            regen = _get_regen_samples(self._rewrite_sampler, text)
+            if not regen:
+                raise RuntimeError("Rewrite model returned no variants.")
+            r0 = regen[0]
+        return self._l2d_stat(text, r0)

miner_lab/subnet_metrics.py

@@ -0,0 +1,156 @@
+"""
+Subnet-style evaluation helpers.
+
+Labels: 0 = human, 1 = AI/LLM. Scores are higher = more likely AI.
+
+FP_score: specificity (true negative rate) at a threshold — rewards not flagging humans as AI.
+  FP_score = TN / (TN + FP) = 1 - FPR
+
+AP: sklearn average_precision_score(y_true, y_score) using continuous scores (ranking quality).
+
+Composite reward: weighted mix of F1, FP_score, AP (default equal 1/3 each).
+  For Subnet32, FP_score is often most important — use reward_weights e.g. (0.25, 0.5, 0.25).
+F1 and FP_score use thresholded preds; AP uses continuous scores (constant across threshold sweep).
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Iterable, List, Optional, Tuple
+
+import numpy as np
+from sklearn.metrics import average_precision_score, f1_score
+
+
+def predict_at_threshold(scores: np.ndarray, threshold: float) -> np.ndarray:
+    scores = np.asarray(scores, dtype=np.float64)
+    return (scores >= threshold).astype(np.int64)
+
+
+def fp_score_at_threshold(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    """Specificity: TN / (TN + FP). y_true/y_pred in {0,1}, positive class = AI."""
+    y_true = np.asarray(y_true, dtype=np.int64)
+    y_pred = np.asarray(y_pred, dtype=np.int64)
+    mask_h = y_true == 0
+    if not np.any(mask_h):
+        return 1.0
+    tn = np.sum((y_pred == 0) & mask_h)
+    fp = np.sum((y_pred == 1) & mask_h)
+    denom = tn + fp
+    return float(tn / denom) if denom > 0 else 0.0
+
+
+def metrics_at_threshold(
+    y_true: np.ndarray,
+    scores: np.ndarray,
+    threshold: float,
+    reward_weights: Tuple[float, float, float] = (1.0 / 3.0, 1.0 / 3.0, 1.0 / 3.0),
+) -> Dict[str, float]:
+    y_true = np.asarray(y_true, dtype=np.int64)
+    scores = np.asarray(scores, dtype=np.float64)
+    y_pred = predict_at_threshold(scores, threshold)
+    f1 = float(f1_score(y_true, y_pred, zero_division=0))
+    fps = fp_score_at_threshold(y_true, y_pred)
+    try:
+        ap = float(average_precision_score(y_true, scores))
+    except ValueError:
+        ap = 0.0
+    wf, wp, wa = reward_weights
+    wsum = wf + wp + wa
+    if wsum <= 0:
+        wsum = 1.0
+    reward = (wf * f1 + wp * fps + wa * ap) / wsum
+    return {"f1": f1, "fp_score": fps, "ap": ap, "reward": reward}
+
+
+def sweep_thresholds(
+    y_true: np.ndarray,
+    scores: np.ndarray,
+    start: float = 0.01,
+    end: float = 0.99,
+    step: float = 0.01,
+    reward_weights: Tuple[float, float, float] = (1.0 / 3.0, 1.0 / 3.0, 1.0 / 3.0),
+) -> Tuple[float, Dict[str, float], List[Dict[str, Any]]]:
+    """
+    Pick threshold maximizing weighted reward (F1, FP_score, AP).
+    AP is constant w.r.t. threshold (uses continuous scores).
+    """
+    y_true = np.asarray(y_true, dtype=np.int64)
+    scores = np.asarray(scores, dtype=np.float64)
+    try:
+        ap_fixed = float(average_precision_score(y_true, scores))
+    except ValueError:
+        ap_fixed = 0.0
+
+    wf, wp, wa = reward_weights
+    wsum = wf + wp + wa
+    if wsum <= 0:
+        wsum = 1.0
+
+    rows: List[Dict[str, Any]] = []
+    best_t, best = 0.5, {"f1": 0.0, "fp_score": 0.0, "ap": ap_fixed, "reward": -1.0}
+    t = start
+    while t <= end + 1e-9:
+        y_pred = predict_at_threshold(scores, t)
+        f1 = float(f1_score(y_true, y_pred, zero_division=0))
+        fps = fp_score_at_threshold(y_true, y_pred)
+        reward = (wf * f1 + wp * fps + wa * ap_fixed) / wsum
+        row = {"threshold": round(t, 4), "f1": f1, "fp_score": fps, "ap": ap_fixed, "reward": reward}
+        rows.append(row)
+        if reward > best["reward"]:
+            best = {"f1": f1, "fp_score": fps, "ap": ap_fixed, "reward": reward}
+            best_t = t
+        t += step
+    return best_t, best, rows
+
+
+def slice_by_key(
+    records: List[dict], key: str
+) -> Dict[str, List[dict]]:
+    out: Dict[str, List[dict]] = {}
+    for r in records:
+        k = str(r.get(key, "unknown"))
+        out.setdefault(k, []).append(r)
+    return out
+
+
+def stress_report(
+    records: List[dict],
+    score_key: str,
+    threshold: float,
+    slice_keys: Optional[Iterable[str]] = None,
+    reward_weights: Tuple[float, float, float] = (1.0 / 3.0, 1.0 / 3.0, 1.0 / 3.0),
+) -> Dict[str, Any]:
+    """
+    records: dicts with 'label' (0/1), score_key (float), optional metadata for slicing.
+    """
+    slice_keys = list(slice_keys or ("domain", "generator_family", "augmentation_type"))
+    y_true = np.array([int(r["label"]) for r in records], dtype=np.int64)
+    scores = np.array([float(r[score_key]) for r in records], dtype=np.float64)
+    overall = metrics_at_threshold(y_true, scores, threshold, reward_weights=reward_weights)
+    report: Dict[str, Any] = {
+        "overall": overall,
+        "n": len(records),
+        "threshold": threshold,
+        "reward_by_slice": {},
+    }
+    for sk in slice_keys:
+        by = slice_by_key(records, sk)
+        per: Dict[str, Any] = {}
+        for name, group in by.items():
+            if len(group) < 2:
+                continue
+            yt = np.array([int(r["label"]) for r in group], dtype=np.int64)
+            sc = np.array([float(r[score_key]) for r in group], dtype=np.float64)
+            per[name] = metrics_at_threshold(yt, sc, threshold, reward_weights=reward_weights)
+        report["reward_by_slice"][sk] = per
+
+    rewards = [v["reward"] for v in report["reward_by_slice"].get("domain", {}).values()]
+    if rewards:
+        report["worst_domain_reward"] = float(min(rewards))
+    all_slice_rewards: List[float] = []
+    for sk, per in report["reward_by_slice"].items():
+        for _, m in per.items():
+            all_slice_rewards.append(m["reward"])
+    if all_slice_rewards:
+        report["worst_slice_reward"] = float(min(all_slice_rewards))
+    return report

miner_lab/sup_scorer.py

@@ -0,0 +1,32 @@
+"""Optional supervised RoBERTa-style classifier (same format as scripts/train_sup_encoder.py)."""
+
+from __future__ import annotations
+
+import os
+
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+
+class SupScorer:
+    def __init__(self, model_dir: str, device_str: str, max_length: int = 256):
+        model_dir = os.path.abspath(model_dir)
+        self.device = torch.device(device_str)
+        self.tokenizer = AutoTokenizer.from_pretrained(model_dir)
+        self.model = AutoModelForSequenceClassification.from_pretrained(model_dir)
+        self.model.to(self.device)
+        self.model.eval()
+        self.max_length = max_length
+
+    def score_text(self, text: str) -> float:
+        enc = self.tokenizer(
+            text,
+            truncation=True,
+            max_length=self.max_length,
+            padding="max_length",
+            return_tensors="pt",
+        ).to(self.device)
+        with torch.no_grad():
+            logits = self.model(**enc).logits
+            prob = torch.softmax(logits, dim=-1)[0, 1].item()
+        return float(prob)

miner_lab/text_features.py

@@ -0,0 +1,14 @@
+import re
+
+
+def count_words(text: str) -> int:
+    if not text or not str(text).strip():
+        return 0
+    return len(str(text).split())
+
+
+def count_sentences(text: str) -> int:
+    if not text or not str(text).strip():
+        return 0
+    parts = re.split(r"[.!?]+(?:\s+|$)", str(text).strip())
+    return max(1, len([p for p in parts if p.strip()]))

miner_lab/weighted_fusion.py

@@ -0,0 +1,154 @@
+"""
+Normalize branch scores on the fit split, then fuse with fixed weights (Subnet32-style).
+
+Default weights: ada 0.45, l2d 0.30, sup 0.25 — Ada drives recall/AP, L2D stabilizes FP.
+"""
+
+from __future__ import annotations
+
+from typing import Dict, List, Mapping, Sequence, Tuple
+
+import numpy as np
+
+DEFAULT_WEIGHTS: Mapping[str, float] = {
+    "ada_score": 0.45,
+    "l2d_score": 0.30,
+    "sup_score": 0.25,
+}
+
+
+def _finite(x) -> bool:
+    try:
+        return np.isfinite(float(x))
+    except (TypeError, ValueError):
+        return False
+
+
+def compute_medians(rows: List[dict], keys: Sequence[str]) -> Dict[str, float]:
+    med: Dict[str, float] = {}
+    for k in keys:
+        vals = [float(r[k]) for r in rows if k in r and _finite(r[k])]
+        med[k] = float(np.median(vals)) if vals else 0.0
+    return med
+
+
+def fit_minmax(rows: List[dict], keys: Sequence[str], medians: Dict[str, float]) -> Dict[str, Dict[str, float]]:
+    """Per-key min/max on fit split (after imputation with medians for missing)."""
+    out: Dict[str, Dict[str, float]] = {}
+    for k in keys:
+        vals = []
+        for r in rows:
+            v = r.get(k)
+            if v is None or not _finite(v):
+                v = medians.get(k, 0.0)
+            vals.append(float(v))
+        if not vals:
+            out[k] = {"min": 0.0, "max": 1.0}
+        else:
+            lo, hi = float(min(vals)), float(max(vals))
+            if hi - lo < 1e-12:
+                out[k] = {"min": lo, "max": hi + 1e-6}
+            else:
+                out[k] = {"min": lo, "max": hi}
+    return out
+
+
+def fit_zscore(rows: List[dict], keys: Sequence[str], medians: Dict[str, float]) -> Dict[str, Dict[str, float]]:
+    """Per-key mean/std on fit split (after imputation)."""
+    out: Dict[str, Dict[str, float]] = {}
+    for k in keys:
+        vals = []
+        for r in rows:
+            v = r.get(k)
+            if v is None or not _finite(v):
+                v = medians.get(k, 0.0)
+            vals.append(float(v))
+        if not vals:
+            out[k] = {"mean": 0.0, "std": 1.0}
+        else:
+            a = np.asarray(vals, dtype=np.float64)
+            mu = float(np.mean(a))
+            sig = float(np.std(a))
+            if sig < 1e-12:
+                sig = 1.0
+            out[k] = {"mean": mu, "std": sig}
+    return out
+
+
+def _norm_minmax(v: float, lo: float, hi: float) -> float:
+    if hi - lo < 1e-12:
+        return 0.5
+    return float(np.clip((v - lo) / (hi - lo), 0.0, 1.0))
+
+
+def _norm_zsigmoid(v: float, mu: float, sig: float) -> float:
+    if sig < 1e-12:
+        return 0.5
+    z = (float(v) - mu) / sig
+    z = float(np.clip(z, -20.0, 20.0))
+    return float(1.0 / (1.0 + np.exp(-z)))
+
+
+def renormalize_weights(weights: Mapping[str, float], active_keys: Sequence[str]) -> Dict[str, float]:
+    w = {k: float(weights[k]) for k in active_keys if k in weights}
+    s = sum(w.values())
+    if s <= 0:
+        n = len(w)
+        return {k: 1.0 / max(n, 1) for k in w}
+    return {k: v / s for k, v in w.items()}
+
+
+def row_branch_values(row: dict, keys: Sequence[str], medians: Dict[str, float]) -> List[float]:
+    vals = []
+    for k in keys:
+        v = row.get(k)
+        if v is None or not _finite(v):
+            v = medians.get(k, 0.0)
+        vals.append(float(v))
+    return vals
+
+
+def fusion_prob_array(
+    rows: List[dict],
+    keys: Sequence[str],
+    medians: Dict[str, float],
+    norm_params: Mapping[str, Mapping[str, float]],
+    weights: Mapping[str, float],
+    norm_method: str,
+) -> np.ndarray:
+    """Vector of fusion scores in ~[0, 1] (weighted sum of normalized branch scores)."""
+    w = renormalize_weights(weights, keys)
+    out = np.zeros(len(rows), dtype=np.float64)
+    for i, r in enumerate(rows):
+        s = 0.0
+        for k in keys:
+            v = r.get(k)
+            if v is None or not _finite(v):
+                v = medians.get(k, 0.0)
+            v = float(v)
+            if norm_method == "minmax":
+                lo = norm_params[k]["min"]
+                hi = norm_params[k]["max"]
+                vn = _norm_minmax(v, lo, hi)
+            elif norm_method == "zscore_sigmoid":
+                mu = norm_params[k]["mean"]
+                sig = norm_params[k]["std"]
+                vn = _norm_zsigmoid(v, mu, sig)
+            else:
+                raise ValueError(f"Unknown norm_method: {norm_method}")
+            s += w[k] * vn
+        out[i] = s
+    return out
+
+
+def fusion_prob_one(
+    row: dict,
+    keys: Sequence[str],
+    medians: Dict[str, float],
+    norm_params: Mapping[str, Mapping[str, float]],
+    weights: Mapping[str, float],
+    norm_method: str,
+) -> float:
+    """Single-row fusion probability (same as one element of fusion_prob_array)."""
+    arr = fusion_prob_array([row], keys, medians, norm_params, weights, norm_method)
+    return float(arr[0])

requirements.txt

@@ -0,0 +1,34 @@
+# PyTorch with NVIDIA GPU: CUDA wheels come from PyTorch’s index (not the default PyPI CPU build).
+# Change cu126 to cu118 or cu128 if needed — match https://pytorch.org/get-started/locally/ and nvidia-smi.
+# CUDA wheels are ~2–3 GB; if pip hits ReadTimeoutError, reinstall with a longer socket timeout, e.g.:
+#   python -m pip install --upgrade pip
+#   python -m pip install --default-timeout=3600 --retries 10 -r requirements.txt
+--extra-index-url https://download.pytorch.org/whl/cu126
+
+torch
+numpy
+# Gemma-2 (google/gemma-2-9b-it) needs transformers>=4.44 (tokenizer + model classes).
+# If textattack warns or breaks, use a separate venv for L2D scoring or upgrade textattack.
+transformers>=4.44.0,<4.50.0
+tokenizers>=0.19.0
+accelerate>=0.33.0
+# L2D checkpoints may include adapter_config fields (e.g. corda_config) from newer PEFT; need >=0.14.
+peft>=0.14.0,<0.20.0
+huggingface_hub>=0.16.0
+# Optional: 4-bit AdaDist on ~12GB GPUs (run_l2d_jsonl.py --load_4bit). Windows may need a matching CUDA wheel.
+bitsandbytes>=0.43.0
+datasets==2.12.0
+# datasets 2.12 expects pyarrow.PyExtensionType (removed in pyarrow 16+); also keeps numpy<2 via pyarrow 15.x.
+pyarrow>=12.0.0,<16.0.0
+fsspec==2023.9.2
+matplotlib
+tqdm
+openai
+nltk
+scikit-learn
+rank-bm25
+selenium
+nltk
+textattack==0.3.10
+# google
+# anthropic

scripts/BSpline.py

@@ -0,0 +1,201 @@
+import numpy as np
+import torch
+from torch import nn
+from helper import *
+
+
+class BSpline(nn.Module):
+    """
+    Class for computing the B-spline funcions b_i(x)
+    and constructing the penality matrix S.
+
+    # Args
+        start: float or int; start of the region
+        end: float or int; end of the region
+        n_bases: int; number of spline bases
+        spline_order: int; spline order
+
+    # Methods
+        - **getS(add_intercept=False)** - Get the penalty matrix S
+              - Args
+                     - **add_intercept**: bool. If true, intercept column is added to the returned matrix.
+              - Returns
+                     - `np.array`, of shape `(n_bases + add_intercept, n_bases + add_intercept)`
+        - **predict(x, add_intercept=False)** - For some x, predict the bn(x) for each base
+              - Args
+                     - **x**: np.array; Vector of dimension 1
+                     - **add_intercept**: bool; If True, intercept column is added to the to the final array
+              - Returns
+                     - `torch.tensor`, of shape `(len(x), n_bases + (add_intercept))`
+    """
+
+    def __init__(self, start=0, end=1, n_bases=10, spline_order=3, intercept=1):
+        super().__init__()
+
+        self.start = start
+        self.end = end
+        self.n_bases = n_bases
+        self.spline_order = spline_order
+        self.add_intercept = True if intercept == 1 else False
+
+        self.knots = get_knots(self.start, self.end, self.n_bases, self.spline_order)
+
+        self.S = get_S(self.n_bases, self.spline_order)
+
+    def __repr__(self):
+        return "BSpline(start={0}, end={1}, n_bases={2}, spline_order={3})".format(
+            self.start, self.end, self.n_bases, self.spline_order
+        )
+
+    def getS(self):
+        """Get the penalty matrix S
+        Returns:
+            torch.tensor, of shape (n_bases + add_intercept, n_bases + add_intercept)
+        """
+        S = self.S
+        if self.add_intercept is True:
+            # S <- cbind(0, rbind(0, S)) # in R
+            zeros = np.zeros_like(S[:1, :])
+            S = np.vstack([zeros, S])
+
+            zeros = np.zeros_like(S[:, :1])
+            S = np.hstack([zeros, S])
+        return S
+
+    def forward(self, x):
+        """For some x, predict the bn(x) for each base
+        Args:
+            x: torch.tensor
+            add_intercept: bool; should we add the intercept to the final array
+        Returns:
+            torch.tensor, of shape (len(x), n_bases + (add_intercept))
+        """
+        # sanity check
+        if x.min() < self.start:
+            raise Warning("x.min() < self.start")
+        if x.max() > self.end:
+            raise Warning("x.max() > self.end")
+
+        return get_X_spline_torch(
+            x=x,
+            knots=self.knots,
+            n_bases=self.n_bases,
+            spline_order=self.spline_order,
+            add_intercept=self.add_intercept,
+        )
+        # return get_X_spline(x=x, knots=self.knots, n_bases=self.n_bases, spline_order=self.spline_order, add_intercept=self.add_intercept)
+
+    def get_config(self):
+        return {
+            "start": self.start,
+            "end": self.end,
+            "n_bases": self.n_bases,
+            "spline_order": self.spline_order,
+        }
+
+    @classmethod
+    def from_config(cls, config):
+        return cls(**config)
+
+
+def _trunc(x, minval=None, maxval=None):
+    """Truncate vector values to have values on range [minval, maxval]"""
+    x = torch.clone(x)
+    if minval != None:
+        x[x < minval] = minval
+    if maxval != None:
+        x[x > maxval] = maxval
+    return x
+
+
+def encodeSplines(x, n_bases=5, spline_order=3, start=None, end=None, warn=True):
+    """Function for the class `EncodeSplines`.
+    Expansion by generating B-spline basis functions for each x
+    and each n (spline-index) with `scipy.interpolate.splev`,
+    based on the pre-placed equidistant knots on [start, end] range.
+
+    # Arguments
+        x: a torch.tensor of positions
+        n_bases int: Number of spline bases.
+        spline_order: 2 for quadratic, 3 for qubic splines
+        start, end: range of values. If None, they are inferred from the data
+        as minimum and maximum value.
+        warn: Show warnings.
+
+    # Returns
+        `torch.tensor` of shape `(x.shape[0], x.shape[1], channels, n_bases)`
+    """
+
+    if len(x.shape) == 1:
+        x = x.reshape((-1, 1))
+
+    if start is None:
+        start = torch.amin(x)  # should be np.nanmin
+    else:
+        if x.min() < start:
+            if warn:
+                print(
+                    "WARNING, x.min() < start for some elements. Truncating them to start: x[x < start] = start"
+                )
+            x = _trunc(x, minval=start)
+    if end is None:
+        end = torch.amax(x)  # should be np.nanmax
+    else:
+        if x.max() > end:
+            if warn:
+                print(
+                    "WARNING, x.max() > end for some elements. Truncating them to end: x[x > end] = end"
+                )
+            x = _trunc(x, maxval=end)
+    bs = BSpline(start, end, n_bases=n_bases, spline_order=spline_order)
+
+    # concatenate x to long
+    assert len(x.shape) == 2
+    n_rows = x.shape[0]
+    n_cols = x.shape[1]
+
+    x_long = x.reshape((-1,))
+
+    # shape = (n_rows * n_cols, n_bases)
+    x_feat = bs.predict(x_long, add_intercept=False)
+
+    x_final = x_feat.reshape((n_rows, n_cols, n_bases))
+    return x_final
+
+
+if __name__ == "__main__":
+    
+    obs = -30 * torch.rand(10)
+    # print(obs)
+    bspline_args = {'start': -32, 'end': 0, 'n_bases': 7, 'spline_order': 2, 'intercept': 1}
+    # bspline_args = {'start': -30, 'end': 0, 'n_bases': 16, 'spline_order': 2}
+    bspline = BSpline(**bspline_args)
+    output = bspline.predict(obs)
+
+    import matplotlib.pyplot as plt
+    x_plot = torch.linspace(bspline.start, bspline.end, 1000)
+    basis_plot = bspline.predict(x_plot)
+
+    plt.figure(figsize=(10, 6))
+    for i in range(bspline.n_bases):
+        plt.plot(x_plot, basis_plot[:, i], label=f'Basis {i + 1}')
+    
+    plt.scatter(obs, torch.zeros_like(obs), color='red', label='Observations')
+    plt.title(f'B-spline Bases (Order {bspline.spline_order}, {bspline.n_bases} Bases)')
+    plt.xlabel('x')
+    plt.ylabel('Basis Value')
+    plt.legend()
+    plt.grid(True)
+    plt.show()
+
+    # beta_hat = torch.tensor([ 0.0000, -0.1320,  0.8180,  0.1750,  0.1690, -0.0490,  0.4600,  0.2010])  # old
+    beta_hat = torch.tensor([-0.0030,  0.0000,  0.0160,  0.1960, -0.0150, -1.0230,  0.4580,  0.3650])
+    predict = basis_plot @ beta_hat
+    plt.figure(figsize=(10, 6))
+    plt.plot(x_plot, predict, label=f'w-function')
+    plt.xlabel('x')
+    plt.ylabel('y')
+    plt.grid(True)
+    plt.show()
+    plt.savefig('w_function.png')
+    

scripts/BiScope/LICENSE

@@ -0,0 +1,402 @@
+Attribution-NonCommercial-NoDerivatives 4.0 International
+
+=======================================================================
+
+Creative Commons Corporation ("Creative Commons") is not a law firm and
+does not provide legal services or legal advice. Distribution of
+Creative Commons public licenses does not create a lawyer-client or
+other relationship. Creative Commons makes its licenses and related
+information available on an "as-is" basis. Creative Commons gives no
+warranties regarding its licenses, any material licensed under their
+terms and conditions, or any related information. Creative Commons
+disclaims all liability for damages resulting from their use to the
+fullest extent possible.
+
+Using Creative Commons Public Licenses
+
+Creative Commons public licenses provide a standard set of terms and
+conditions that creators and other rights holders may use to share
+original works of authorship and other material subject to copyright
+and certain other rights specified in the public license below. The
+following considerations are for informational purposes only, are not
+exhaustive, and do not form part of our licenses.
+
+     Considerations for licensors: Our public licenses are
+     intended for use by those authorized to give the public
+     permission to use material in ways otherwise restricted by
+     copyright and certain other rights. Our licenses are
+     irrevocable. Licensors should read and understand the terms
+     and conditions of the license they choose before applying it.
+     Licensors should also secure all rights necessary before
+     applying our licenses so that the public can reuse the
+     material as expected. Licensors should clearly mark any
+     material not subject to the license. This includes other CC-
+     licensed material, or material used under an exception or
+     limitation to copyright. More considerations for licensors:
+    wiki.creativecommons.org/Considerations_for_licensors
+
+     Considerations for the public: By using one of our public
+     licenses, a licensor grants the public permission to use the
+     licensed material under specified terms and conditions. If
+     the licensor's permission is not necessary for any reason--for
+     example, because of any applicable exception or limitation to
+     copyright--then that use is not regulated by the license. Our
+     licenses grant only permissions under copyright and certain
+     other rights that a licensor has authority to grant. Use of
+     the licensed material may still be restricted for other
+     reasons, including because others have copyright or other
+     rights in the material. A licensor may make special requests,
+     such as asking that all changes be marked or described.
+     Although not required by our licenses, you are encouraged to
+     respect those requests where reasonable. More considerations
+     for the public:
+    wiki.creativecommons.org/Considerations_for_licensees
+
+=======================================================================
+
+Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
+International Public License
+
+By exercising the Licensed Rights (defined below), You accept and agree
+to be bound by the terms and conditions of this Creative Commons
+Attribution-NonCommercial-NoDerivatives 4.0 International Public
+License ("Public License"). To the extent this Public License may be
+interpreted as a contract, You are granted the Licensed Rights in
+consideration of Your acceptance of these terms and conditions, and the
+Licensor grants You such rights in consideration of benefits the
+Licensor receives from making the Licensed Material available under
+these terms and conditions.
+
+
+Section 1 -- Definitions.
+
+  a. Adapted Material means material subject to Copyright and Similar
+     Rights that is derived from or based upon the Licensed Material
+     and in which the Licensed Material is translated, altered,
+     arranged, transformed, or otherwise modified in a manner requiring
+     permission under the Copyright and Similar Rights held by the
+     Licensor. For purposes of this Public License, where the Licensed
+     Material is a musical work, performance, or sound recording,
+     Adapted Material is always produced where the Licensed Material is
+     synched in timed relation with a moving image.
+
+  b. Copyright and Similar Rights means copyright and/or similar rights
+     closely related to copyright including, without limitation,
+     performance, broadcast, sound recording, and Sui Generis Database
+     Rights, without regard to how the rights are labeled or
+     categorized. For purposes of this Public License, the rights
+     specified in Section 2(b)(1)-(2) are not Copyright and Similar
+     Rights.
+
+  c. Effective Technological Measures means those measures that, in the
+     absence of proper authority, may not be circumvented under laws
+     fulfilling obligations under Article 11 of the WIPO Copyright
+     Treaty adopted on December 20, 1996, and/or similar international
+     agreements.
+
+  d. Exceptions and Limitations means fair use, fair dealing, and/or
+     any other exception or limitation to Copyright and Similar Rights
+     that applies to Your use of the Licensed Material.
+
+  e. Licensed Material means the artistic or literary work, database,
+     or other material to which the Licensor applied this Public
+     License.
+
+  f. Licensed Rights means the rights granted to You subject to the
+     terms and conditions of this Public License, which are limited to
+     all Copyright and Similar Rights that apply to Your use of the
+     Licensed Material and that the Licensor has authority to license.
+
+  g. Licensor means the individual(s) or entity(ies) granting rights
+     under this Public License.
+
+  h. NonCommercial means not primarily intended for or directed towards
+     commercial advantage or monetary compensation. For purposes of
+     this Public License, the exchange of the Licensed Material for
+     other material subject to Copyright and Similar Rights by digital
+     file-sharing or similar means is NonCommercial provided there is
+     no payment of monetary compensation in connection with the
+     exchange.
+
+  i. Share means to provide material to the public by any means or
+     process that requires permission under the Licensed Rights, such
+     as reproduction, public display, public performance, distribution,
+     dissemination, communication, or importation, and to make material
+     available to the public including in ways that members of the
+     public may access the material from a place and at a time
+     individually chosen by them.
+
+  j. Sui Generis Database Rights means rights other than copyright
+     resulting from Directive 96/9/EC of the European Parliament and of
+     the Council of 11 March 1996 on the legal protection of databases,
+     as amended and/or succeeded, as well as other essentially
+     equivalent rights anywhere in the world.
+
+  k. You means the individual or entity exercising the Licensed Rights
+     under this Public License. Your has a corresponding meaning.
+
+
+Section 2 -- Scope.
+
+  a. License grant.
+
+       1. Subject to the terms and conditions of this Public License,
+          the Licensor hereby grants You a worldwide, royalty-free,
+          non-sublicensable, non-exclusive, irrevocable license to
+          exercise the Licensed Rights in the Licensed Material to:
+
+            a. reproduce and Share the Licensed Material, in whole or
+               in part, for NonCommercial purposes only; and
+
+            b. produce and reproduce, but not Share, Adapted Material
+               for NonCommercial purposes only.
+
+       2. Exceptions and Limitations. For the avoidance of doubt, where
+          Exceptions and Limitations apply to Your use, this Public
+          License does not apply, and You do not need to comply with
+          its terms and conditions.
+
+       3. Term. The term of this Public License is specified in Section
+          6(a).
+
+       4. Media and formats; technical modifications allowed. The
+          Licensor authorizes You to exercise the Licensed Rights in
+          all media and formats whether now known or hereafter created,
+          and to make technical modifications necessary to do so. The
+          Licensor waives and/or agrees not to assert any right or
+          authority to forbid You from making technical modifications
+          necessary to exercise the Licensed Rights, including
+          technical modifications necessary to circumvent Effective
+          Technological Measures. For purposes of this Public License,
+          simply making modifications authorized by this Section 2(a)
+          (4) never produces Adapted Material.
+
+       5. Downstream recipients.
+
+            a. Offer from the Licensor -- Licensed Material. Every
+               recipient of the Licensed Material automatically
+               receives an offer from the Licensor to exercise the
+               Licensed Rights under the terms and conditions of this
+               Public License.
+
+            b. No downstream restrictions. You may not offer or impose
+               any additional or different terms or conditions on, or
+               apply any Effective Technological Measures to, the
+               Licensed Material if doing so restricts exercise of the
+               Licensed Rights by any recipient of the Licensed
+               Material.
+
+       6. No endorsement. Nothing in this Public License constitutes or
+          may be construed as permission to assert or imply that You
+          are, or that Your use of the Licensed Material is, connected
+          with, or sponsored, endorsed, or granted official status by,
+          the Licensor or others designated to receive attribution as
+          provided in Section 3(a)(1)(A)(i).
+
+  b. Other rights.
+
+       1. Moral rights, such as the right of integrity, are not
+          licensed under this Public License, nor are publicity,
+          privacy, and/or other similar personality rights; however, to
+          the extent possible, the Licensor waives and/or agrees not to
+          assert any such rights held by the Licensor to the limited
+          extent necessary to allow You to exercise the Licensed
+          Rights, but not otherwise.
+
+       2. Patent and trademark rights are not licensed under this
+          Public License.
+
+       3. To the extent possible, the Licensor waives any right to
+          collect royalties from You for the exercise of the Licensed
+          Rights, whether directly or through a collecting society
+          under any voluntary or waivable statutory or compulsory
+          licensing scheme. In all other cases the Licensor expressly
+          reserves any right to collect such royalties, including when
+          the Licensed Material is used other than for NonCommercial
+          purposes.
+
+
+Section 3 -- License Conditions.
+
+Your exercise of the Licensed Rights is expressly made subject to the
+following conditions.
+
+  a. Attribution.
+
+       1. If You Share the Licensed Material, You must:
+
+            a. retain the following if it is supplied by the Licensor
+               with the Licensed Material:
+
+                 i. identification of the creator(s) of the Licensed
+                    Material and any others designated to receive
+                    attribution, in any reasonable manner requested by
+                    the Licensor (including by pseudonym if
+                    designated);
+
+                ii. a copyright notice;
+
+               iii. a notice that refers to this Public License;
+
+                iv. a notice that refers to the disclaimer of
+                    warranties;
+
+                 v. a URI or hyperlink to the Licensed Material to the
+                    extent reasonably practicable;
+
+            b. indicate if You modified the Licensed Material and
+               retain an indication of any previous modifications; and
+
+            c. indicate the Licensed Material is licensed under this
+               Public License, and include the text of, or the URI or
+               hyperlink to, this Public License.
+
+          For the avoidance of doubt, You do not have permission under
+          this Public License to Share Adapted Material.
+
+       2. You may satisfy the conditions in Section 3(a)(1) in any
+          reasonable manner based on the medium, means, and context in
+          which You Share the Licensed Material. For example, it may be
+          reasonable to satisfy the conditions by providing a URI or
+          hyperlink to a resource that includes the required
+          information.
+
+       3. If requested by the Licensor, You must remove any of the
+          information required by Section 3(a)(1)(A) to the extent
+          reasonably practicable.
+
+
+Section 4 -- Sui Generis Database Rights.
+
+Where the Licensed Rights include Sui Generis Database Rights that
+apply to Your use of the Licensed Material:
+
+  a. for the avoidance of doubt, Section 2(a)(1) grants You the right
+     to extract, reuse, reproduce, and Share all or a substantial
+     portion of the contents of the database for NonCommercial purposes
+     only and provided You do not Share Adapted Material;
+
+  b. if You include all or a substantial portion of the database
+     contents in a database in which You have Sui Generis Database
+     Rights, then the database in which You have Sui Generis Database
+     Rights (but not its individual contents) is Adapted Material; and
+
+  c. You must comply with the conditions in Section 3(a) if You Share
+     all or a substantial portion of the contents of the database.
+
+For the avoidance of doubt, this Section 4 supplements and does not
+replace Your obligations under this Public License where the Licensed
+Rights include other Copyright and Similar Rights.
+
+
+Section 5 -- Disclaimer of Warranties and Limitation of Liability.
+
+  a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
+     EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
+     AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
+     ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
+     IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
+     WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
+     PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
+     ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
+     KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
+     ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
+
+  b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
+     TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
+     NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
+     INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
+     COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
+     USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
+     ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
+     DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
+     IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
+
+  c. The disclaimer of warranties and limitation of liability provided
+     above shall be interpreted in a manner that, to the extent
+     possible, most closely approximates an absolute disclaimer and
+     waiver of all liability.
+
+
+Section 6 -- Term and Termination.
+
+  a. This Public License applies for the term of the Copyright and
+     Similar Rights licensed here. However, if You fail to comply with
+     this Public License, then Your rights under this Public License
+     terminate automatically.
+
+  b. Where Your right to use the Licensed Material has terminated under
+     Section 6(a), it reinstates:
+
+       1. automatically as of the date the violation is cured, provided
+          it is cured within 30 days of Your discovery of the
+          violation; or
+
+       2. upon express reinstatement by the Licensor.
+
+     For the avoidance of doubt, this Section 6(b) does not affect any
+     right the Licensor may have to seek remedies for Your violations
+     of this Public License.
+
+  c. For the avoidance of doubt, the Licensor may also offer the
+     Licensed Material under separate terms or conditions or stop
+     distributing the Licensed Material at any time; however, doing so
+     will not terminate this Public License.
+
+  d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
+     License.
+
+
+Section 7 -- Other Terms and Conditions.
+
+  a. The Licensor shall not be bound by any additional or different
+     terms or conditions communicated by You unless expressly agreed.
+
+  b. Any arrangements, understandings, or agreements regarding the
+     Licensed Material not stated herein are separate from and
+     independent of the terms and conditions of this Public License.
+
+
+Section 8 -- Interpretation.
+
+  a. For the avoidance of doubt, this Public License does not, and
+     shall not be interpreted to, reduce, limit, restrict, or impose
+     conditions on any use of the Licensed Material that could lawfully
+     be made without permission under this Public License.
+
+  b. To the extent possible, if any provision of this Public License is
+     deemed unenforceable, it shall be automatically reformed to the
+     minimum extent necessary to make it enforceable. If the provision
+     cannot be reformed, it shall be severed from this Public License
+     without affecting the enforceability of the remaining terms and
+     conditions.
+
+  c. No term or condition of this Public License will be waived and no
+     failure to comply consented to unless expressly agreed to by the
+     Licensor.
+
+  d. Nothing in this Public License constitutes or may be interpreted
+     as a limitation upon, or waiver of, any privileges and immunities
+     that apply to the Licensor or You, including from the legal
+     processes of any jurisdiction or authority.
+
+=======================================================================
+
+Creative Commons is not a party to its public
+licenses. Notwithstanding, Creative Commons may elect to apply one of
+its public licenses to material it publishes and in those instances
+will be considered the “Licensor.” The text of the Creative Commons
+public licenses is dedicated to the public domain under the CC0 Public
+Domain Dedication. Except for the limited purpose of indicating that
+material is shared under a Creative Commons public license or as
+otherwise permitted by the Creative Commons policies published at
+creativecommons.org/policies, Creative Commons does not authorize the
+use of the trademark "Creative Commons" or any other trademark or logo
+of Creative Commons without its prior written consent including,
+without limitation, in connection with any unauthorized modifications
+to any of its public licenses or any other arrangements,
+understandings, or agreements concerning use of licensed material. For
+the avoidance of doubt, this paragraph does not form part of the
+public licenses.
+
+Creative Commons may be contacted at creativecommons.org.

scripts/BiScope/biscope_utils.py

@@ -0,0 +1,225 @@
+import os
+import json
+import time
+import pickle
+from tqdm import tqdm
+import numpy as np
+import torch
+from torch.nn import CrossEntropyLoss
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Minimal model zoo mapping model keys to pretrained model names.
+MODEL_ZOO = {
+    'llama2-7b': 'meta-llama/Llama-2-7b-chat-hf',
+    'llama2-13b': 'meta-llama/Llama-2-13b-chat-hf',
+    'gemma-2b': 'google/gemma-1.1-2b-it',
+    'gemma-7b': 'google/gemma-1.1-7b-it', 
+    # 'llama3-8b': 'meta-llama/Meta-Llama-3-8B-Instruct',
+    'gpt2-xl': 'gpt2-xl',
+    'qwen-7b': 'Qwen/Qwen2.5-7B',
+    'llama3-8b': 'meta-llama/Meta-Llama-3-8B',
+    'mistralai-7b': 'mistralai/Mistral-7B-Instruct-v0.2',
+}
+
+# Prompt templates for text completion.
+COMPLETION_PROMPT_ONLY = "Complete the following text: "
+COMPLETION_PROMPT = "Given the summary:\n{prompt}\n Complete the following text: "
+
+def generate(model, tokenizer, input_ids, trigger_length, target_length):
+    """
+    Generate additional tokens using the model's generation API.
+    
+    Parameters:
+      model: the language model for generation.
+      tokenizer: associated tokenizer.
+      input_ids: input token IDs (either 1D or 2D).
+      trigger_length: the length of the prompt (number of tokens to skip in the output).
+      target_length: the number of new tokens to generate.
+      
+    Returns:
+      Generated tokens (as a 2D tensor) after removing the trigger tokens.
+    """
+    config = model.generation_config
+    config.max_new_tokens = target_length
+    # If input_ids is 1D, add a batch dimension; otherwise, assume it's already 2D.
+    if input_ids.dim() == 1:
+        input_ids = input_ids.to(model.device).unsqueeze(0)
+    else:
+        input_ids = input_ids.to(model.device)
+    # Create an attention mask of the same shape.
+    attn_masks = torch.ones(input_ids.shape, device=input_ids.device)
+    # Generate new tokens.
+    out = model.generate(
+        input_ids, 
+        attention_mask=attn_masks,
+        generation_config=config,
+        pad_token_id=tokenizer.pad_token_id
+    )[0]
+    # Return output tokens after the prompt (slice along dimension 1).
+    return out[trigger_length:]
+
+
+def compute_fce_loss(logits, targets, text_slice):
+    """
+    Compute the FCE loss by shifting indices by 1.
+    Returns a NumPy array of loss values.
+    """
+    loss = CrossEntropyLoss(reduction='none')(
+        logits[0, text_slice.start-1:text_slice.stop-1, :],
+        targets
+    )
+    return loss.detach().cpu().numpy()
+
+def compute_bce_loss(logits, targets, text_slice):
+    """
+    Compute the BCE loss without shifting indices.
+    Returns a NumPy array of loss values.
+    """
+    loss = CrossEntropyLoss(reduction='none')(
+        logits[0, text_slice, :],
+        targets
+    )
+    return loss.detach().cpu().numpy()
+
+def detect_single_sample(args, model, tokenizer, summary_model, summary_tokenizer, sample, device='cuda'):
+    """
+    Process a sample by generating a summary-based prompt, tokenizing (with clipping),
+    obtaining model outputs, and computing loss-based features (FCE and BCE).
+    Returns a list of loss features computed over 10 segments.
+    """
+    # Generate the summary-based prompt.
+    if 'gpt-' in args.summary_model:
+        from openai import OpenAI
+        openai_key = os.environ.get('OPENAI_API_KEY')
+        if not openai_key:
+            raise ValueError("OPENAI_API_KEY not found in environment.")
+        client = OpenAI(api_key=openai_key)
+        from tenacity import (
+            retry,
+            stop_after_attempt,
+            wait_random_exponential,
+        )
+        @retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))
+        def openai_backoff(client, **kwargs):
+            return client.chat.completions.create(**kwargs)
+        summary_input = f"generate a very short and concise summary for the following text, just the summary: {sample}"
+        response = openai_backoff(client, model=args.summary_model,
+                                  messages=[{"role": "user", "content": summary_input}])
+        summary_text = response.choices[0].message.content.strip()
+        # if '"""' in summary_text:
+        #     summary_text = summary_text.split('"""')[-1]
+        prompt_text = COMPLETION_PROMPT.format(prompt=summary_text)
+    elif args.summary_model in MODEL_ZOO:
+        summary_input = f"Write a title for this text: {sample}\nJust output the title:"
+        summary_ids = summary_tokenizer(summary_input, return_tensors='pt',
+                                        max_length=args.sample_clip, truncation=True).input_ids.to(device)
+        summary_ids = summary_ids[:, 1:]  # Remove start token.
+        gen_ids = generate(summary_model, summary_tokenizer, summary_ids, summary_ids.shape[1], 64)
+        summary_text = summary_tokenizer.decode(gen_ids, skip_special_tokens=True).strip().split('\n')[0]
+        prompt_text = COMPLETION_PROMPT.format(prompt=summary_text)
+    else:
+        prompt_text = COMPLETION_PROMPT_ONLY
+
+    # Tokenize the prompt and sample with token-level clipping.
+    prompt_ids = tokenizer(prompt_text, return_tensors='pt').input_ids.to(device)
+    text_ids = tokenizer(sample, return_tensors='pt', max_length=args.sample_clip, truncation=True).input_ids.to(device)
+    combined_ids = torch.cat([prompt_ids, text_ids], dim=1)
+    text_slice = slice(prompt_ids.shape[1], combined_ids.shape[1])
+    outputs = model(input_ids=combined_ids)
+    logits = outputs.logits
+    targets = combined_ids[0][text_slice]
+    
+    # Compute loss features from FCE and BCE losses.
+    fce_loss = compute_fce_loss(logits, targets, text_slice)
+    bce_loss = compute_bce_loss(logits, targets, text_slice)
+    features = []
+    for p in range(1, 10):
+        split = len(fce_loss) * p // 10
+        features.extend([
+            np.mean(fce_loss[split:]), np.max(fce_loss[split:]), 
+            np.min(fce_loss[split:]), np.std(fce_loss[split:]),
+            np.mean(bce_loss[split:]), np.max(bce_loss[split:]), 
+            np.min(bce_loss[split:]), np.std(bce_loss[split:])
+        ])
+    return features
+
+def data_generation(args, out_dir, task, generative_model, base_dir):
+    """
+    Generate loss-based features for both human and GPT samples and save them to disk.
+    
+    Parameters:
+      out_dir: Output directory.
+      task: Task name (e.g., Arxiv, Code, Essay).
+      generative_model: Key for the GPT samples.
+      
+    Returns:
+      The output directory.
+    """
+    # Load summary model and its tokenizer if specified.
+    if args.summary_model in MODEL_ZOO:
+        summary_model = AutoModelForCausalLM.from_pretrained(
+            MODEL_ZOO[args.summary_model],
+            torch_dtype=torch.float16,
+            device_map='auto',
+            cache_dir=args.cache_dir,
+        ).eval()
+        summary_tokenizer = AutoTokenizer.from_pretrained(
+            MODEL_ZOO[args.summary_model], padding_side='left', cache_dir=args.cache_dir,
+        )
+        summary_tokenizer.pad_token = summary_tokenizer.eos_token
+    else:
+        summary_model, summary_tokenizer = None, None
+
+    # Load detection model and its tokenizer.
+    if args.detect_model in MODEL_ZOO:
+        model = AutoModelForCausalLM.from_pretrained(
+            MODEL_ZOO[args.detect_model],
+            torch_dtype=torch.float16,
+            device_map='auto', 
+            cache_dir=args.cache_dir,
+        ).eval()
+        tokenizer = AutoTokenizer.from_pretrained(
+            MODEL_ZOO[args.detect_model], padding_side='left', cache_dir=args.cache_dir,
+        )
+        tokenizer.pad_token = tokenizer.eos_token
+    else:
+        raise ValueError("Unknown detection model")
+
+    if getattr(args, "use_hf_dataset", False):
+        # Load from Hugging Face dataset.
+        from datasets import load_dataset
+        ds = load_dataset("HanxiGuo/BiScope_Data", split="train")
+        paraphrased_flag = False
+
+        # Filter human data: use rows where task matches, and source is 'human'.
+        human_data = ds.filter(lambda x: x["task"] == task and x["source"].lower() == "human")
+        human_data = [s["text"] for s in human_data]
+
+        # Filter GPT-generated data: use rows where task and paraphrased flag match and source matches generative_model.
+        # (Assumes that the GPT-generated samples have source equal to the provided generative_model string.)
+        gpt_data = ds.filter(lambda x: x["task"] == task and x["paraphrased"] == paraphrased_flag and x["source"].lower() == generative_model.lower())
+        gpt_data = [s["text"] for s in gpt_data]
+    else:
+        
+        # Load human data. Human data do not have paraphrased version, so only use the normal data.
+        with open(f'{base_dir}/{task}_{generative_model}.raw_data.json', 'r') as f:
+            dataset = json.load(f)
+
+        human_data = dataset['original']
+        gpt_data = dataset['sampled']
+    
+    # Define the human features file path internally.
+    human_feat_path = os.path.join(out_dir, f"{task}_human_features.pkl")
+    
+    # Generate and save human features.
+    human_features = [detect_single_sample(args, model, tokenizer, summary_model, summary_tokenizer, s, device='cuda') for s in tqdm(human_data)]
+    with open(human_feat_path, 'wb') as f:
+        pickle.dump(human_features, f)
+    
+    # Generate and save GPT features.
+    gpt_feat_path = os.path.join(out_dir, f"{task}_GPT_features.pkl")
+    gpt_features = [detect_single_sample(args, model, tokenizer, summary_model, summary_tokenizer, s, device='cuda') for s in tqdm(gpt_data)]
+    with open(gpt_feat_path, 'wb') as f:
+        pickle.dump(gpt_features, f)
+    
+    return out_dir

scripts/ImBD/.gitignore

@@ -0,0 +1,165 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+ckpt/*/
+logs/*/
+models/*/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

scripts/ImBD/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

scripts/ImBD/dataset.py

@@ -0,0 +1,106 @@
+from torch.utils.data import Dataset
+import json
+from utils import load_training_data
+
+class CustomDataset(Dataset):
+    def __init__(self, data_json_dir):
+        with open(data_json_dir, 'r') as f:
+            data_json = json.load(f)
+        self.data = self.process_data(data_json)
+
+    def __len__(self):
+        return len(self.data['original'])
+
+    def __getitem__(self, index):
+        original_text = self.data['original'][index]
+        sampled_text = self.data['sampled'][index]
+
+        return {
+            'text': [original_text, sampled_text],
+            'label': [0, 1]  # Original label is 0, Sampled label is 1
+        }
+
+    def process_data(self, data_json):
+        processed_data = {
+            'original': data_json['original'],
+            'sampled': data_json['sampled']
+        }
+
+        return processed_data
+
+class CustomDataset_rewrite(Dataset):
+    def __init__(self, data_json_dir):
+        self.data_json_dir = data_json_dir
+        if "&" in data_json_dir:
+            data_name_list = data_json_dir.split('&')
+            data_json = load_training_data(data_name_list)
+        else:
+            with open(data_json_dir+'.raw_data.json', 'r') as f:
+                data_json = json.load(f)
+        self.data = self.process_data(data_json)
+
+    def __len__(self):
+        return len(self.data['original'])
+
+    def __getitem__(self, index):
+        original_text = self.data['original'][index]
+        sampled_text = self.data['sampled'][index]
+
+        return original_text, sampled_text
+
+    def process_data(self, data_json):
+        if "pubmed" in self.data_json_dir:
+            processed_data = {
+                'original': [qa.split("Answer:")[1].strip() for qa in data_json['original']],
+                'sampled': [qa.split("Answer:")[1].strip() for qa in data_json['sampled']]
+            }
+        else:
+            processed_data = {
+                'original': data_json['original'],
+                'sampled': data_json['sampled']
+            }
+
+        return processed_data
+
+class CustomDataset_split(Dataset):
+    def __init__(self, data_json_dir, split='train', val_ratio=0.2):
+        with open(data_json_dir, 'r') as f:
+            data_json = json.load(f)
+        self.data = self.process_data(data_json)
+        
+        total_size = len(self.data['original'])
+        
+        if val_ratio == 0: 
+            self.indices = [i for i in range(total_size)]
+            return
+            
+        # Compute step size for stratified sampling
+        step_size = int(1 / val_ratio)
+
+        val_indices = list(range(0, total_size, step_size))
+        train_indices = [i for i in range(total_size) if i not in val_indices]
+        # print(val_indices)
+        # print(train_indices)
+        if split == 'train':
+            self.indices = train_indices
+        elif split == 'val':
+            self.indices = val_indices
+        else:
+            raise ValueError("split must be either 'train' or 'val'")
+
+    def __len__(self):
+        return len(self.indices)
+
+    def __getitem__(self, index):
+        actual_index = self.indices[index]
+        original_text = self.data['original'][actual_index]
+        sampled_text = self.data['rewritten'][actual_index]
+        return original_text, sampled_text
+
+    def process_data(self, data_json):
+        processed_data = {
+            'original': data_json['original'],
+            'rewritten': data_json['rewritten']
+        }
+
+        return processed_data

scripts/ImBD/engine.py

@@ -0,0 +1,184 @@
+# -*- coding: utf-8 -*-
+from torch.utils.data import DataLoader
+import tqdm
+from torch.cuda.amp import GradScaler, autocast
+import torch.nn.functional as F
+from torch import nn
+import torch
+import numpy as np
+from torch.utils.tensorboard import SummaryWriter
+import datetime
+import os
+import json
+from metrics import get_roc_metrics, get_precision_recall_metrics
+
+from torch.optim.lr_scheduler import CosineAnnealingLR
+import time
+from utils import GpuMem
+try:
+    from transformers import AdamW
+except:
+    from torch.optim import AdamW
+
+def evaluate_model_SPO(model, data, DEVICE):
+    model.to(DEVICE)
+    model.eval()
+    loss = 0
+    eval_loader = DataLoader(data, batch_size=1, shuffle=False)
+    epoch_crit_train_original, epoch_crit_train_sampled = [],[]
+    start_time = time.time()
+    with torch.no_grad():
+        for batch in tqdm.tqdm(eval_loader, desc="Evaluating"):
+            text = batch
+            output = model(text)
+            loss += output['loss'].item()
+            epoch_crit_train_original.extend(output['crit'][1].tolist())
+            epoch_crit_train_sampled.extend(output['crit'][3].tolist())
+            
+        print(f"Total time: {time.time() - start_time:.4f}s")
+        avg_loss = loss / len(eval_loader)
+        fpr, tpr, roc_auc = get_roc_metrics(epoch_crit_train_original, epoch_crit_train_sampled)
+        p, r, pr_auc = get_precision_recall_metrics(epoch_crit_train_original, epoch_crit_train_sampled)
+    
+    # print(f"val_loss: {avg_loss:.6f}")
+    print(f"val_ROC_AUC: {roc_auc:.4f}, PR AUC: {pr_auc:.4f}")
+    print(f"val_Real_mean/std: {np.mean(epoch_crit_train_original):.2f}/{np.std(epoch_crit_train_original):.2f}, val_Samples_mean/std: {np.mean(epoch_crit_train_sampled):.2f}/{np.std(epoch_crit_train_sampled):.2f}")
+    print("="*10)
+    
+    results_dict = {
+        "name": "imbd",
+        'info': {'n_samples': len(epoch_crit_train_original)},
+        'predictions': {'real': epoch_crit_train_original, 
+                        'samples': epoch_crit_train_sampled},
+        'metrics': {'roc_auc': roc_auc, 'fpr': fpr, 'tpr': tpr},
+        'pr_metrics': {'pr_auc': pr_auc, 'precision': p, 'recall': r},
+    }
+    return results_dict
+
+
+def fine_tune_ours(model, data, DEVICE, ckpt_dir='./ckpt', args=None):
+    current_time = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+    writer = SummaryWriter(log_dir=f"./scripts/ImBD/logs/{args.task_name}_spo_lr_{args.lr}_beta_{args.beta}_a_{args.a}_{current_time}/train_ai_detection")
+
+    train_loader = DataLoader(data[0], batch_size=1, shuffle=True)
+    epochs = args.epochs
+    optimizer = AdamW(model.parameters(), lr=args.lr)
+    scheduler = CosineAnnealingLR(optimizer, T_max=len(train_loader) * epochs, eta_min=0,
+                                  last_epoch=-1)
+
+    scaler = GradScaler()
+    model.to(DEVICE)
+
+    # Number of iterations for gradient accumulation
+    accumulation_steps = args.a
+    epoch_losses, i, loss = [], 0, torch.tensor(0.0).to(DEVICE)
+    epoch_crit_train_original, epoch_crit_train_sampled = [],[]
+    start_time = time.time()
+    for epoch in range(epochs):
+        optimizer.zero_grad()
+        start_time = time.time()
+        for batch in tqdm.tqdm(train_loader, desc=f"Fine-tuning: {epoch} epoch"):
+            text = batch
+            scheduler.step()
+            with autocast():
+                outputs_1 = model(text)
+                epoch_crit_train_original.extend(outputs_1['crit'][1].tolist())
+                epoch_crit_train_sampled.extend(outputs_1['crit'][3].tolist())
+                loss += (outputs_1['loss'].to(torch.float32)) / accumulation_steps
+            
+            
+            if ((i + 1) % accumulation_steps) == 0:
+                scaler.scale(loss).backward()
+                scaler.step(optimizer)
+                optimizer.zero_grad()
+                scaler.update()
+                writer.add_scalar('Loss/train', loss.item(), i)  
+                epoch_losses.append(loss.item())
+                loss = torch.tensor(0.0).to(DEVICE)
+            epoch_losses.append(loss.item())
+            i += 1
+        print(f"Total time: {time.time() - start_time:.4f}s") 
+        fpr, tpr, roc_auc = get_roc_metrics(epoch_crit_train_original, epoch_crit_train_sampled)
+        p, r, pr_auc = get_precision_recall_metrics(epoch_crit_train_original, epoch_crit_train_sampled)
+        
+        print(f"ROC AUC: {roc_auc:.4f}, PR AUC: {pr_auc:.4f}")
+        print(f"Real mean/std: {np.mean(epoch_crit_train_original):.2f}/{np.std(epoch_crit_train_original):.2f}, Samples mean/std: {np.mean(epoch_crit_train_sampled):.2f}/{np.std(epoch_crit_train_sampled):.2f}")
+        epoch_avg_loss = np.mean(epoch_losses)
+        
+        writer.add_scalar('Loss/epoch', epoch_avg_loss, epoch)
+        writer.add_scalar('ROC_AUC/epoch', roc_auc, epoch)
+        writer.add_scalar('PR_AUC/epoch', pr_auc, epoch)
+        writer.add_scalar('Real_mean/epoch',np.mean(epoch_crit_train_original),epoch)
+        writer.add_scalar('Real_std/epoch',np.std(epoch_crit_train_original),epoch)
+        writer.add_scalar('Sampled_mean/epoch',np.mean(epoch_crit_train_sampled),epoch)
+        writer.add_scalar('Sampled_std/epoch',np.std(epoch_crit_train_sampled),epoch)
+        epoch_crit_train_original, epoch_crit_train_sampled = [],[] # reset crit
+        print(f"\nAverage Loss for Epoch {epoch}: {epoch_avg_loss}")
+            
+    # if not os.path.exists(ckpt_dir):
+    #     os.makedirs(ckpt_dir)
+    # model.save_pretrained(ckpt_dir)
+    # print(f"Saved finetuned model to {os.path.join(ckpt_dir, 'ours-finetuned.pth')}")
+    
+    writer.close()
+    return model
+
+
+def run(
+    model, 
+    data, 
+    DEVICE,
+    args,
+    ckpt_dir='./ckpt',
+    ):
+    
+    if args.ebt or args.eval_only:
+        print("Evaluating model before tuning...")
+        d = evaluate_model_SPO(model, data[1], DEVICE)
+        if args.SPOtrained:
+            output_path = f"{args.output_file}.imbd.json"
+        else:
+            method_name=args.base_model.split("_")[-1]
+            output_path = f"{args.output_file}.{method_name}.json"
+        with open(output_path, "w") as j:
+            json.dump(d,j)
+        print(f"Results saved to {output_path}.")
+    if args.eval_only:
+        return
+
+    tracker = GpuMem()
+    print('Fine-tuning model...')
+    start = time.perf_counter()
+    with tracker:
+        model = fine_tune_ours(
+            model, 
+            data,
+            DEVICE=DEVICE, 
+            ckpt_dir=ckpt_dir,
+            args=args
+        )
+    pre_time = time.perf_counter() - start
+    pre_memory = tracker.memory_usage()
+    
+    if args.eval_after_train:
+        print("Evaluating model after tuning...")
+        start = time.perf_counter()
+        with tracker:
+            d = evaluate_model_SPO(model, data[1], DEVICE)
+        eval_time = time.perf_counter() - start
+        eval_time = eval_time / (len(data[1]) << 1)
+        eval_memory = tracker.memory_usage()
+        d['compute_info'] = {'pre_time': pre_time, 'eval_time': eval_time, 
+                             'pre_memory': pre_memory, 'eval_memory': eval_memory,}
+        if args.SPOtrained:
+            output_path = f"{args.output_file}.imbd.json"
+        else:
+            method_name=args.base_model.split("_")[-1]
+            output_path = f"{args.output_file}.{method_name}.json"
+        with open(output_path, "w") as j:
+            json.dump(d, j)
+        print(f"Results saved to {output_path}.")
+    
+
+
+

scripts/ImBD/evaluation.py

@@ -0,0 +1,82 @@
+import numpy as np
+import torch
+import argparse
+from .spo import get_sampling_discrepancy_analytic
+from transformers import AutoTokenizer
+from peft import AutoPeftModelForCausalLM
+import time
+from metrics import get_roc_metrics, get_precision_recall_metrics
+import tqdm
+from torch.utils.data import DataLoader
+from .dataset import CustomDataset_rewrite
+import os
+import json
+
+def run(args):
+    
+    # Generate the inference checkpoint from trained model
+    # from spo import ComputeScore
+    # model = ComputeScore("gpt-neo-2.7B", "gpt-neo-2.7B", cache_dir=args.cache_dir)
+    # model.from_pretrained(args.from_pretrained)
+    # scoring_model = model.scoring_model
+    # scoring_tokenizer = model.scoring_tokenizer
+    # scoring_model.save_pretrained('models/ImBD-inference')
+    # scoring_tokenizer.save_pretrained('models/ImBD-inference')
+    
+    model_name = "models/ImBD-inference"
+    print('Loading model')
+    start_time = time.time()
+    scoring_model = AutoPeftModelForCausalLM.from_pretrained(model_name) # Make sure you have downloaded the gpt-neo-2.7b and place it at `models` folder.
+    scoring_tokenizer = AutoTokenizer.from_pretrained(model_name)
+    scoring_tokenizer.pad_token = scoring_tokenizer.eos_token
+    scoring_model.to(args.device)
+    scoring_model.eval()
+    print(f'Done. ({time.time()-start_time:.2f}s)')
+    
+    criterion_fn = get_sampling_discrepancy_analytic
+    print('Loading dataset')
+    start_time = time.time()
+    val_data = CustomDataset_rewrite(data_json_dir=args.eval_dataset)
+    print(f'Done. ({time.time()-start_time:.2f}s)')
+    print(f"Evaluating on {args.eval_dataset.split('/')[-1]}")
+    evaluate_model(scoring_model, scoring_tokenizer, criterion_fn, val_data, args)
+    
+        
+def evaluate_model(model, tokenizer, criterion_fn, data, args):
+    eval_loader = DataLoader(data, batch_size=1, shuffle=False)
+    epoch_crit_train_original, epoch_crit_train_sampled = [],[]
+    start_time = time.time()
+    with torch.no_grad():
+        for batch in tqdm.tqdm(eval_loader, desc=f"Evaluating {args.eval_dataset.split('/')[-1]}"):
+            output = []
+            for text in batch:
+                tokenized = tokenizer(text, truncation=True, return_tensors="pt", padding=True, return_token_type_ids=False).to(args.device)
+                labels = tokenized.input_ids[:, 1:]
+                with torch.no_grad():
+                    logits_score = model(**tokenized).logits[:, :-1]
+                    logits_ref = logits_score
+                    crit, _ = criterion_fn(logits_ref, logits_score, labels)
+                crit = crit.cpu().numpy().item()
+                output.append(crit)
+                # output.append(inv_crit)
+            crit_original, crit_sampled = output
+            epoch_crit_train_original.append(crit_original)
+            epoch_crit_train_sampled.append(crit_sampled)
+            
+        print(f"Total time: {time.time() - start_time:.4f}s")
+        fpr, tpr, roc_auc = get_roc_metrics(epoch_crit_train_original, epoch_crit_train_sampled)
+        p, r, pr_auc = get_precision_recall_metrics(epoch_crit_train_original, epoch_crit_train_sampled)
+        os.makedirs("./evaluation", exist_ok=True)
+        with open(os.path.join("./evaluation", args.eval_dataset.split("/")[-1].replace("raw_data", str(roc_auc)[:7])),'w') as f:
+            json.dump({"fpr":fpr, "tpr":tpr, "roc_auc":roc_auc, "crit_real":epoch_crit_train_original, "crit_sampled":epoch_crit_train_sampled, "pr_auc":pr_auc}, f)
+        print(f"ROC AUC:{roc_auc}")
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--device', type=str, default="cuda")
+    parser.add_argument('--cache_dir', type=str, default="../cache")
+    parser.add_argument('--eval_dataset', type=str, default="./data/polish/gpt-4o/xsum_polish_gpt-4o.raw_data.json")
+    # parser.add_argument('--from_pretrained', type=str, default='./ckpt/ai_detection_polish_500_spo_lr_0.0001_beta_0.05_a_1')
+    args = parser.parse_args()
+
+    run(args)

scripts/ImBD/inference.py

@@ -0,0 +1,117 @@
+import numpy as np
+import torch
+import os
+import glob
+import argparse
+import json
+from .spo import get_sampling_discrepancy_analytic
+from transformers import AutoTokenizer
+from peft import AutoPeftModelForCausalLM
+import time
+
+# estimate the probability according to the distribution of our test results on GPT-3.5 and GPT-4
+class ProbEstimator:
+    def __init__(self, args):
+        self.tasks = ["polish", "generate", "rewrite", "expand"] if args.task == "all" else [args.task]
+        self.real_crits = {"polish":[], "generate":[], "rewrite":[], "expand":[]}
+        self.fake_crits = {"polish":[], "generate":[], "rewrite":[], "expand":[]}
+        for task in self.tasks:
+            for result_file in glob.glob(os.path.join(args.ref_path, task, '*.json')):
+                with open(result_file, 'r') as fin:
+                    res = json.load(fin)
+                    self.real_crits[task].extend(res['crit_real'])
+                    self.fake_crits[task].extend(res['crit_sampled'])
+        print(f'ProbEstimator: total {sum([len(self.real_crits[task]) for task in self.tasks]) * 2} samples.')
+
+    def crit_to_prob(self, crit):
+        real_crits = []
+        fake_crits = []
+        for task in self.tasks:
+            real_crits.extend(self.real_crits[task])
+            fake_crits.extend(self.fake_crits[task])
+        offset = np.sort(np.abs(np.array(real_crits + fake_crits) - crit))[100]
+        cnt_real = np.sum((np.array(real_crits) > crit - offset) & (np.array(real_crits) < crit + offset))
+        cnt_fake = np.sum((np.array(fake_crits) > crit - offset) & (np.array(fake_crits) < crit + offset))
+        return cnt_fake / (cnt_real + cnt_fake)
+
+    def crit_to_prob_detail(self, crit):
+        probs = []
+        for task in self.tasks:
+            real_crits = self.real_crits[task]
+            fake_crits = self.fake_crits[task]
+            offset = np.sort(np.abs(np.array(real_crits + fake_crits) - crit))[100]
+            cnt_real = np.sum((np.array(real_crits) > crit - offset) & (np.array(real_crits) < crit + offset))
+            cnt_fake = np.sum((np.array(fake_crits) > crit - offset) & (np.array(fake_crits) < crit + offset))
+            probs.append(cnt_fake / (cnt_real + cnt_fake))
+        return probs
+
+# run interactive local inference
+def run(args):
+    
+    # Generate the inference checkpoint from trained model
+    # from spo import ComputeScore
+    # model = ComputeScore("gpt-neo-2.7B", "gpt-neo-2.7B", cache_dir=args.cache_dir)
+    # model.from_pretrained("ckpt/ai_detection_polish_500_spo_lr_0.0001_beta_0.05_a_1")
+    # scoring_model = model.scoring_model
+    # scoring_tokenizer = model.scoring_tokenizer
+    # scoring_model.save_pretrained('models/ImBD-inference')
+    # scoring_tokenizer.save_pretrained('models/ImBD-inference')
+    
+    print('Loading model')
+    start_time = time.time()
+    scoring_model = AutoPeftModelForCausalLM.from_pretrained('models/ImBD-inference') # Make sure you have downloaded the gpt-neo-2.7b and place it at `models` folder.
+    scoring_tokenizer = AutoTokenizer.from_pretrained('models/ImBD-inference')
+    scoring_tokenizer.pad_token = scoring_tokenizer.eos_token
+    scoring_model.to(args.device)
+    scoring_model.eval()
+    print(f'Done. ({time.time()-start_time:.2f}s)')
+    
+    criterion_fn = get_sampling_discrepancy_analytic
+    prob_estimator = ProbEstimator(args)
+    
+    # input text
+    print('Local demo for ImBD, where the longer text has more reliable result.')
+    print('To view detail results for all tasks, set `--detail` to True.')
+    print('To view all-in-one results, set `--task` to `all`. (Not accurate enough)')
+    print('')
+    while True:
+        print("Please enter your text: (Press Enter twice to start processing)")
+        lines = []
+        while True:
+            line = input()
+            if len(line) == 0:
+                break
+            lines.append(line)
+        text = "\n".join(lines)
+        if len(text) == 0:
+            break
+        # evaluate text
+        tokenized = scoring_tokenizer(text, truncation=True, return_tensors="pt", padding=True, return_token_type_ids=False).to(args.device)
+        labels = tokenized.input_ids[:, 1:]
+        with torch.no_grad():
+            logits_score = scoring_model(**tokenized).logits[:, :-1]
+            logits_ref = logits_score
+            crit, _ = criterion_fn(logits_ref, logits_score, labels)
+        # estimate the probability of machine generated text
+        crit = crit.cpu().numpy().item()
+        if args.detail:
+            probs = prob_estimator.crit_to_prob_detail(crit)
+            print(f'ImBD criterion is {crit:.4f}, suggesting that the text has a probability of',sep=" ")
+            for task, prob in zip(prob_estimator.tasks, probs):
+                print(f'{prob * 100:.0f}% to be machine-{task},',sep=" ")
+        else:
+            prob = prob_estimator.crit_to_prob(crit)
+            print(f'ImBD criterion is {crit:.4f}, suggesting that the text has a probability of {prob * 100:.0f}% to be machine-{args.task}.')
+        print()
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--ref_path', type=str, default="./local_infer_ref")
+    parser.add_argument('--device', type=str, default="cuda")
+    parser.add_argument('--cache_dir', type=str, default="../cache")
+    parser.add_argument('--task', type=str, default="generate", choices=["polish", "generate", "rewrite", "expand", "all"])
+    parser.add_argument('--detail', type=bool, default=False)
+    parser.add_argument('--from_pretrained', type=str, default='./ckpt/ai_detection_polish_500_spo_lr_0.0001_beta_0.05_a_1')
+    args = parser.parse_args()
+
+    run(args)

scripts/ImBD/spo.py

@@ -0,0 +1,189 @@
+import torch
+from torch import nn
+
+from peft import get_peft_model, LoraConfig, TaskType, AutoPeftModelForCausalLM
+import os
+from .utils_spo import calculate_SPO_loss
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import time
+
+def from_pretrained(cls, model_name, kwargs, cache_dir):
+    # use local model if it exists
+    if "/" in model_name:
+        local_path = os.path.join(cache_dir, model_name.split("/")[1])
+    else:
+        local_path = os.path.join(cache_dir, model_name)
+
+    if os.path.exists(local_path):
+        return cls.from_pretrained(local_path, **kwargs)
+    return cls.from_pretrained(model_name, **kwargs, cache_dir=cache_dir, device_map='auto')
+
+model_fullnames = {  
+    'gpt-neo-2.7B': 'EleutherAI/gpt-neo-2.7B',
+    'gpt-j-6B': 'EleutherAI/gpt-j-6B',
+    'qwen-7b': 'Qwen/Qwen2.5-7B',
+    'mistralai-7b': 'mistralai/Mistral-7B-Instruct-v0.2',
+    'llama3-8b': 'meta-llama/Meta-Llama-3-8B',
+}
+float16_models = ['gpt-j-6B', 'gpt-neox-20b', 'qwen-7b', 'mistralai-7b', 'llama3-8b']
+
+
+def get_model_fullname(model_name):
+    return model_fullnames[model_name] if model_name in model_fullnames else model_name
+
+def load_tokenizer(model_name, for_dataset, cache_dir):
+    model_fullname = get_model_fullname(model_name)
+    optional_tok_kwargs = {}
+    if "facebook/opt-" in model_fullname:
+        print("Using non-fast tokenizer for OPT")
+        optional_tok_kwargs['fast'] = False
+    if for_dataset in ['pubmed']:
+        optional_tok_kwargs['padding_side'] = 'left'
+    else:
+        optional_tok_kwargs['padding_side'] = 'right'
+    base_tokenizer = from_pretrained(AutoTokenizer, model_fullname, optional_tok_kwargs, cache_dir=cache_dir)
+    if base_tokenizer.pad_token_id is None:
+        base_tokenizer.pad_token_id = base_tokenizer.eos_token_id
+        if '13b' in model_fullname:
+            base_tokenizer.pad_token_id = 0
+    return base_tokenizer
+
+
+def get_sampling_discrepancy_analytic(logits_ref, logits_score, labels):
+
+    if logits_ref.size(-1) != logits_score.size(-1):
+        vocab_size = min(logits_ref.size(-1), logits_score.size(-1))
+        logits_ref = logits_ref[:, :, :vocab_size]
+        logits_score = logits_score[:, :, :vocab_size]
+
+    labels = labels.unsqueeze(-1) if labels.ndim == logits_score.ndim - 1 else labels
+    lprobs_score = torch.log_softmax(logits_score, dim=-1)
+    probs_ref = torch.softmax(logits_ref, dim=-1)
+    
+    log_likelihood = lprobs_score.gather(dim=-1, index=labels).squeeze(-1)
+    mean_ref = (probs_ref * lprobs_score).sum(dim=-1)
+    var_ref = (probs_ref * torch.square(lprobs_score)).sum(dim=-1) - torch.square(mean_ref)
+    discrepancy = (log_likelihood.sum(dim=-1) - mean_ref.sum(dim=-1)) / var_ref.sum(dim=-1).sqrt()
+    
+    return discrepancy, log_likelihood.sum(dim=-1)
+
+class ComputeScore(nn.Module):
+    def __init__(self, scoring_model_name, reference_model_name, SPOtrained=True, SPO_beta=0.5, dataset='xsum', device='cuda', cache_dir='./models'):
+        super().__init__()
+        self.device = device
+        self.reference_model_name = get_model_fullname(reference_model_name)
+        self.scoring_model_name = get_model_fullname(scoring_model_name)
+        self.beta = SPO_beta
+        
+        def load_model(model_name, device, cache_dir, SPOtrained=True):
+            model_fullname = get_model_fullname(model_name)
+            print(f'Loading model {model_fullname}...')
+            model_kwargs = {}
+            if model_name in float16_models:
+                model_kwargs.update(dict(torch_dtype=torch.float16))
+            if 'gpt-j' in model_name:
+                model_kwargs.update(dict(revision='float16'))
+            if SPOtrained:
+                model = from_pretrained(AutoModelForCausalLM, model_fullname, model_kwargs, cache_dir)
+            else: # Load ablation finetuned model
+                model = from_pretrained(AutoPeftModelForCausalLM, model_fullname, model_kwargs, cache_dir)
+            print('Moving model to GPU...', end='', flush=True)
+            start = time.time()
+            model.to(device)
+            print(f'DONE ({time.time() - start:.2f}s)')
+            return model
+        
+        # load model
+        self.scoring_tokenizer = load_tokenizer(scoring_model_name, dataset, cache_dir)
+        scoring_model = load_model(scoring_model_name, device, cache_dir, SPOtrained)
+
+        self.peft_config = LoraConfig(
+            task_type=TaskType.CAUSAL_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1
+        )
+
+        reference_model = load_model(reference_model_name, device, cache_dir, SPOtrained)
+        
+        if SPOtrained: 
+            self.scoring_model = get_peft_model(scoring_model, self.peft_config)
+            self.reference_model = get_peft_model(reference_model, self.peft_config)
+        else: 
+            self.scoring_model = scoring_model
+            self.reference_model = reference_model
+            
+        
+        self.reference_tokenizer = load_tokenizer(reference_model_name, dataset, cache_dir)
+             
+        self.criterion_fn = get_sampling_discrepancy_analytic
+        self.forward = self.forward_SPO
+
+        self.scoring_model.print_trainable_parameters()
+
+
+    def print_gradient_requirement(self):
+        for name, param in self.named_parameters():
+            gradient_requirement = 'Requires Grad' if param.requires_grad else 'Does not require grad'
+            color_code = '\033[92m' if param.requires_grad else '\033[91m'  # Green for requires grad, red for does not require grad
+            reset_color = '\033[0m'  # Reset color after printing
+            print(f"{name}: {color_code}{gradient_requirement}{reset_color}")
+
+    def register_no_grad(self, module_names):
+        for name, param in self.named_parameters():
+            for selected_module in module_names:
+                # print(selected_module, name)
+                if selected_module in name:
+                    param.requires_grad = False
+
+    def save_pretrained(self, save_directory):
+        """
+        Save the model's state_dict to the specified directory.
+        """
+        os.makedirs(save_directory, exist_ok=True)
+        torch.save(self.state_dict(), os.path.join(save_directory, "model.bin"))
+
+    def from_pretrained(self, load_directory):
+        """
+        Load the model's state_dict from the specified directory.
+        """
+        if not os.path.exists(load_directory):
+            raise ValueError(f"Directory {load_directory} does not exist.")
+
+        self.load_state_dict(torch.load(os.path.join(load_directory, "model.bin"), map_location=self.device))
+
+    def get_SPO_input(self, tokenized=None ,text=[""], labels=[""], training_module=False):
+        if training_module:
+            logits_score = self.scoring_model(tokenized.input_ids, attention_mask=tokenized.attention_mask).logits[:,:-1,:]
+            logits_ref = logits_score
+            if self.reference_model_name != self.scoring_model_name:
+                tokenized = self.reference_tokenizer(text, return_tensors="pt", padding=True, return_token_type_ids=False, add_special_tokens=True, return_attention_mask=True).to(self.device)
+                assert torch.all(tokenized.input_ids[:, 1:] == labels), "Tokenizer is mismatch."
+                logits_ref = self.reference_model(tokenized.input_ids).logits[:,:-1,:]
+            crit, SPO_input  = self.criterion_fn(logits_ref, logits_score, labels)
+        else:
+            with torch.no_grad(): # get reference
+                if self.reference_model_name != self.scoring_model_name:
+                    tokenized = self.reference_tokenizer(text, return_tensors="pt", padding=True, return_token_type_ids=False ,add_special_tokens=True, return_attention_mask=True).to(self.device)
+                    assert torch.all(tokenized.input_ids[:, 1:] == labels), "Tokenizer is mismatch."
+                logits_score = self.reference_model(tokenized.input_ids, attention_mask=tokenized.attention_mask).logits[:,:-1,:] # shape: [bsz, sentence_len, dim]
+                logits_ref = logits_score
+                crit, SPO_input = self.criterion_fn(logits_ref, logits_score, labels)
+        return crit, SPO_input, logits_score
+
+    def forward_SPO(self, text):
+        original_text = text[0]
+        sampled_text = text[1]
+        
+        tokenized = self.scoring_tokenizer(original_text, return_tensors="pt", padding=True, return_token_type_ids=False).to(self.device)
+        labels = tokenized.input_ids[:, 1:] # shape: [bsz, sentence_len]
+        ref_original_crit, ref_disprefered_logprob, ref_original_logits_score = self.get_SPO_input(tokenized,original_text,labels)
+        train_original_crit, train_disprefered_logprob, train_original_logits_score = self.get_SPO_input(tokenized,original_text,labels,training_module=True)
+        
+        tokenized = self.scoring_tokenizer(sampled_text, return_tensors="pt", padding=True, return_token_type_ids=False).to(self.device)
+        labels = tokenized.input_ids[:, 1:]
+        ref_sampled_crit, ref_prefered_logprob, ref_sampled_logits_score = self.get_SPO_input(tokenized,sampled_text,labels)
+        train_sampled_crit, train_prefered_logprob, train_sampled_logits_score = self.get_SPO_input(tokenized,sampled_text,labels,training_module=True)
+        
+        SPOloss, prefered_relative_logprob, disprefered_relative_logprob, reward_accuracies, reward_margins = calculate_SPO_loss(train_prefered_logprob,train_disprefered_logprob,ref_prefered_logprob,ref_disprefered_logprob,beta=self.beta)
+        output = dict(crit=[ref_original_crit, train_original_crit, ref_sampled_crit, train_sampled_crit], loss=SPOloss)
+        return output
+
+

scripts/ImBD/utils_spo.py

@@ -0,0 +1,14 @@
+import torch.nn.functional as F
+
+def calculate_SPO_loss(model_prefered_logprob, model_disprefered_logprob,
+                       ref_prefered_logprob, ref_disprefered_logprob,
+                       beta=0.5):
+
+    prefered_relative_logprob = model_prefered_logprob - ref_prefered_logprob
+    disprefered_relative_logprob = model_disprefered_logprob - ref_disprefered_logprob
+
+    reward_accuracies = (prefered_relative_logprob > disprefered_relative_logprob).float().mean(dim=-1)
+    reward_margins = (prefered_relative_logprob - disprefered_relative_logprob).mean(dim=-1)
+    loss = -F.logsigmoid(beta * (prefered_relative_logprob - disprefered_relative_logprob)).mean(dim=-1)
+    
+    return loss, prefered_relative_logprob.mean(dim=-1), disprefered_relative_logprob.mean(dim=-1), reward_accuracies, reward_margins

scripts/TextFluoroscopy/utils_fluoroscopy.py

@@ -0,0 +1,149 @@
+from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModel
+import torch
+import torch.nn.functional as F
+import os
+import json
+from tqdm import tqdm
+import pickle
+import huggingface_hub 
+huggingface_hub.login("hf_xxxxx")  # replace hf_xxxxx with your actual token
+
+def last_token_pool(last_hidden_states,
+                 attention_mask):
+    left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0])
+    if left_padding:
+        return last_hidden_states[:, -1]
+    else:
+        sequence_lengths = attention_mask.sum(dim=1) - 1
+        batch_size = last_hidden_states.shape[0]
+        return last_hidden_states[torch.arange(batch_size, device='cpu'), sequence_lengths]
+    
+def load_model(model_name, cache_dir):
+    tokenizer = AutoTokenizer.from_pretrained(
+        model_name,       # <-- Hub ID, *not* your cache path
+        cache_dir=cache_dir,         # where to store/download files
+        trust_remote_code=True,       # allow loading the repo’s custom code
+        use_auth_token=True,
+    )
+    tokenizer.pad_token = tokenizer.eos_token
+    model = AutoModelForCausalLM.from_pretrained(
+        model_name, 
+        trust_remote_code=True, 
+        device_map='auto',
+        use_auth_token=True, 
+        cache_dir=cache_dir
+    )
+    return tokenizer, model
+
+def get_kl(model, tokenizer, input_texts, max_length, device):
+    batch_dict = tokenizer(input_texts, max_length=max_length, 
+                           padding=True, truncation=True, return_tensors='pt').to(device)
+    with torch.no_grad():
+        outputs = model(**batch_dict, output_hidden_states=True)
+        last_logits = model.lm_head(outputs.hidden_states[-1]).squeeze()
+        first_logits = model.lm_head(outputs.hidden_states[0]).squeeze()
+    kls = []
+    for i in range(1, len(outputs.hidden_states) - 1):
+        with torch.no_grad():
+            middle_logits = model.lm_head(outputs.hidden_states[i]).squeeze()
+        kls.append(F.kl_div(F.log_softmax(middle_logits, dim=-1), F.softmax(first_logits, dim=-1), reduction='batchmean').item() +
+                   F.kl_div(F.log_softmax(middle_logits, dim=-1), F.softmax(last_logits, dim=-1), reduction='batchmean').item())
+    return kls
+
+def compute_kl_feat(model, tokenizer, file_name, save_dir, max_length, device):
+    test_datasets = {}
+    if not os.path.exists(save_dir):
+        os.makedirs(save_dir)
+    data_name = file_name.split('/')[-1]
+    if not os.path.exists(save_dir + data_name + '.pkl'):
+        print(file_name)
+        test_datasets[file_name] = {'data': [], 'label': []}
+        with open(file_name + '.raw_data.json', 'r') as f:
+            data = json.load(f)
+        kls = []
+        n_samples = len(data['original'])
+        # n_samples = 30
+        for idx in tqdm(range(n_samples)):
+            human_text = data['original'][idx]
+            kl = get_kl(model, tokenizer, [human_text], max_length, device)
+            kls.append(kl)
+            llm_text = data['sampled'][idx]
+            kl = get_kl(model, tokenizer, [llm_text], max_length, device)
+            kls.append(kl)
+            # if len(kls) >= 300:
+            #     break
+        print(save_dir + data_name + '.pkl')
+        pickle.dump(kls, open(save_dir + data_name + '.pkl', 'wb'))
+
+def load_model2(model_name, cache_dir):
+    tokenizer = AutoTokenizer.from_pretrained(
+        model_name,       # <-- Hub ID, *not* your cache path
+        cache_dir=cache_dir,         # where to store/download files
+        trust_remote_code=True,       # allow loading the repo’s custom code
+        use_auth_token=True,
+    )
+    tokenizer.pad_token = tokenizer.eos_token
+    model = AutoModel.from_pretrained(
+        model_name, 
+        trust_remote_code=True, 
+        device_map='auto',
+        use_auth_token=True, 
+        cache_dir=cache_dir
+    )
+    return tokenizer, model
+
+def get_all_embedding(model, tokenizer, input_texts, max_length, device):
+    batch_dict = tokenizer(input_texts, max_length=max_length, padding=True, truncation=True, return_tensors='pt').to(device)
+    with torch.no_grad():
+        outputs = model(**batch_dict, output_hidden_states=True)
+    all_embed = [last_token_pool(outputs.hidden_states[i].cpu(), batch_dict['attention_mask']) for i in range(len(outputs.hidden_states))]
+    all_embed = torch.concat(all_embed, 1).cpu()
+    return all_embed
+
+def compute_embedding(model, tokenizer, file_name, save_dir, max_length, device):
+    test_datasets = {}
+    if not os.path.exists(save_dir):
+        os.makedirs(save_dir)
+    data_name = file_name.split('/')[-1]
+    if not os.path.exists(save_dir + data_name + '.pt'):
+        print(file_name)
+        test_datasets[file_name] = {'data':[], 'label':[]}
+        with open(file_name + '.raw_data.json', 'r') as f:
+            data = json.load(f)
+        embeddings = []
+        n_samples = len(data['original'])
+        # n_samples = 30
+        for idx in tqdm(range(n_samples)):
+            human_text = data['original'][idx]
+            embedding = get_all_embedding(model, tokenizer, [human_text], max_length, device)
+            embeddings.append(embedding)
+            llm_text = data['sampled'][idx]
+            embedding = get_all_embedding(model, tokenizer, [llm_text], max_length, device)
+            embeddings.append(embedding)
+            # if len(embeddings) >=300:
+            #     break
+        embeddings = torch.cat(embeddings, dim=0)
+        print('embedding shape: ', embeddings.shape)
+        print(save_dir + data_name +'.pt')
+        torch.save(embeddings, save_dir + data_name + '.pt')
+
+
+if __name__ == '__main__':
+    device = 'cuda'
+    max_length = 512
+    
+    model_name = "Alibaba-NLP/gte-Qwen1.5-7B-instruct"
+    # cache_dir = '/root/autodl-tmp/cache'
+    cache_dir = '../../../cache'
+    
+    which_embedding='gte-qwen_KL_with_first_and_last_layer'
+    save_dir = f'save/{which_embedding}/'
+    data_dir = 'dataset/processed_data/'
+    tokenizer, model = load_model(model_name, cache_dir)
+    compute_kl_feat(model, tokenizer, data_dir, save_dir, max_length, device)
+    
+    which_embedding='gte-qwen_all_embedding'
+    save_dir = f'save/{which_embedding}/save_embedding/'
+    data_dir = 'dataset/processed_data/'
+    tokenizer, model = load_model2(model_name, cache_dir)
+    compute_embedding(model, tokenizer, data_dir, save_dir, max_length, device)

scripts/attack_data.py

@@ -0,0 +1,166 @@
+# Copyright (c) Guangsheng Bao.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+import argparse
+import json
+import logging
+import random
+import re
+import sys
+import time
+import nltk
+import numpy as np
+import torch
+from tqdm import tqdm
+
+import os
+os.chdir(os.path.dirname(os.path.abspath(__file__)))
+
+logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
+
+
+def bt_translation(src, browser):
+    zh2en_url = f'https://translate.google.com/?hl=zh&sl=en&tl=zh-CN&text={src}&op=translate'
+    browser.get(zh2en_url)  # 访问相对应链接 browser.close#关闭浏览器
+    time.sleep(random.randint(1, 2))
+    browser.find_element_by_xpath(
+        '//*[@id="yDmH0d"]/c-wiz/div/div[2]/c-wiz/div[2]/c-wiz/div[1]/div[2]/div[3]/c-wiz[1]/span/span/div/div[2]/div[1]').send_keys(
+        src)
+    browser.refresh()
+    # time.sleep(50)
+    time.sleep(random.randint(2, 3))
+    text = browser.find_element_by_xpath(
+        '/html/body/c-wiz/div/div[2]/c-wiz/div[2]/c-wiz/div[1]/div[2]/div[3]/c-wiz[2]/div/div[8]/div/div[1]/span[1]').text
+    en_text = text.replace("翻譯搜尋結果\n", "").replace("\n", "")
+    en2zh_url = f'https://translate.google.com/?hl=zh&sl=zh-CN&tl=en&text={en_text}&op=translate'
+    browser.get(en2zh_url)  # 访问相对应链接 browser.close#关闭浏览器
+    time.sleep(random.randint(1, 2))
+    browser.refresh()
+    time.sleep(random.randint(2, 3))
+    text = browser.find_element_by_xpath(
+        '/html/body/c-wiz/div/div[2]/c-wiz/div[2]/c-wiz/div[1]/div[2]/div[3]/c-wiz[2]/div/div[8]/div/div[1]/span[1]').text
+    tgt = text.replace("翻譯搜尋結果\n", "").replace("\n", "")
+    return tgt
+
+
+def read_data(json_path):
+    with open(json_path, 'r', encoding="utf-8") as f:
+        data = json.load(f)
+    return data
+
+
+def count_sentences_in_paragraph(paragraph):
+    sentences = nltk.sent_tokenize(paragraph)
+    return len(sentences)
+
+
+def save_json_data(data, path):
+    with open(path, "w", encoding="utf-8") as outfile:
+        json.dump(data, outfile, ensure_ascii=False, indent=4)
+
+
+def replace_line_breaks(s):
+    s = re.sub('\n', ' ', s)
+    return s
+
+
+def truncate_to_last_sentence(s):
+    # 从字符串尾部向前找句号的位置
+    last_period = s.rfind('.') or s.rfind('!') or s.rfind('?')
+    # 如果找到句号，就截断到这个位置（包括句号）
+    if last_period != -1:
+        s = s[:last_period + 1]
+    return s
+
+
+def check_paragraphs(texts):
+    if count_sentences_in_paragraph(texts) >= 4:
+        return True
+    else:
+        return False
+
+def load_data(input_file):
+    data_file = f"{input_file}.raw_data.json"
+    with open(data_file, "r") as fin:
+        data = json.load(fin)
+        print(f"Raw data loaded from {data_file}")
+    return data
+
+def save_data(output_file, data):
+    # write the data to a json file in the save folder
+    data_file = f"{output_file}.raw_data.json"
+    with open(data_file, "w") as fout:
+        json.dump(data, fout, indent=4)
+        print(f"Raw data written into {data_file}")
+
+
+def run(args):
+    # set seed
+    random.seed(args.seed)
+    torch.manual_seed(args.seed)
+    np.random.seed(args.seed)
+
+    # load data
+    data = load_data(args.input_path)
+
+    # perturbation attacks prepare
+    if "perturbation" in args.method:
+        from textattack.augmentation import TextBuggerAugmenter
+        from textattack.augmentation import TextFoolerAugmenter
+        from textattack.augmentation import DeepWordBugAugmenter
+        word_augmenter = TextFoolerAugmenter()
+        character_augmenter = DeepWordBugAugmenter()
+        word_character_augmenter = TextBuggerAugmenter()
+
+    human_key = "original"
+    llm_key = 'sampled'
+
+    n_samples = len(data)
+    for i in tqdm(range(n_samples)):
+        human = data[human_key][i]
+        llm = data[llm_key][i]
+
+        # llms selection
+        if "perturbation" in args.method:
+            humans = count_sentences_in_paragraph(human)
+            llms = count_sentences_in_paragraph(llm)
+            for attack in ["perturbation_character", "perturbation_word", "perturbation_sent"]:
+                if attack == "perturbation_character":
+                    augmenter = character_augmenter
+                elif attack == "perturbation_word":
+                    augmenter = word_augmenter
+                elif attack == "perturbation_sent":
+                    augmenter = word_character_augmenter
+                else:
+                    raise ValueError(f"{attack} is not in perturbation_attacks")
+
+                try:
+                    # final_data = []
+                    # for d in range(humans):
+                    #     final_data.append(augmenter.augment(d)[0])
+                    # human_result = ' '.join(final_data)
+                    # data[human_key][i] = human_result
+                    # logging.info(f"{attack} human finished")
+
+                    final_data = []
+                    for d in range(llms):
+                        final_data.append(augmenter.augment(d)[0])
+                    llm_result = ' '.join(final_data)
+                    data[llm_key][i] = llm_result
+                    logging.info(f"{attack} llm finished")
+
+                except Exception as e:
+                    logging.info(f"error: {e}")
+                    pass
+        
+    save_data(args.output_file, data)
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--input_path', required=False, default="./exp_main/data/xsum_opt-2.7b", type=str)
+    parser.add_argument('--output_path', required=False, default="./exp_main/data/xsum_opt-2.7b", type=str)
+    parser.add_argument('--method', default="perturbation_sent", type=str, choices=["perturbation_character", "perturbation_word", "perturbation_sent"], required=False)
+    parser.add_argument('--seed', default=2023, type=int, required=False)
+    args = parser.parse_args()
+    run(args)