README.md

---
license: mit
datasets:
- saiteja33/DAMASHA
language:
- en
base_model:
- FacebookAI/roberta-base
- answerdotai/ModernBERT-base
pipeline_tag: token-classification
---

# DAMASHA-MAS: Mixed-Authorship Adversarial Segmentation (Token Classification)

This repository contains a **token-classification model** trained on the **DAMASHA-MAS** benchmark, introduced in:

> **DAMASHA: Detecting AI in Mixed Adversarial Texts via Segmentation with Human-interpretable Attribution**  

The model aims to **segment mixed human–AI text** at *token level* – i.e., decide for each token whether it was written by a *human* or an *LLM*, even under **syntactic adversarial attacks**.

- **Base encoders:**  
  - [`FacebookAI/roberta-base`](https://huggingface.co/FacebookAI/roberta-base) 
  - [`answerdotai/ModernBERT-base`](https://huggingface.co/answerdotai/ModernBERT-base) 
- **Architecture (high level):** RoBERTa + ModernBERT feature fusion → BiGRU + CRF with the **Info-Mask** gating mechanism from the paper.  
- **Task:** Token classification (binary authorship: human vs AI).  
- **Language:** English  
- **License (this model):** MIT  
- **Training data license:** CC-BY-4.0 via the DAMASHA dataset.  

If you use this model, **please also cite the DAMASHA paper and dataset** (see Citation section).

---

## 1. Model Highlights

- **Fine-grained mixed-authorship detection**  
  Predicts authorship **per token**, allowing reconstruction of human vs AI **spans** in long documents.  

- **Adversarially robust**  
  Trained and evaluated on **syntactically attacked texts** (misspelling, Unicode substitutions, invisible characters, punctuation swaps, case perturbations, and “all-mixed” attacks).  

- **Human-interpretable Info-Mask**  
  The architecture incorporates **stylometric features** (perplexity, POS density, punctuation density, lexical diversity, readability) via an **Info-Mask** module that gates token representations in an interpretable way. 

- **Strong reported performance (from the paper)**  
  On DAMASHA-MAS, the **RMC\*** model (RoBERTa + ModernBERT + CRF + Info-Mask) achieves:  
  - **Token-level**: Accuracy / Precision / Recall / F1 ≈ **0.98**  
  - **Span-level (strict)**: SBDA ≈ **0.45**, SegPre ≈ **0.41**  
  - **Span-level (relaxed IoU ≥ 0.5)**: ≈ **0.82**  

> ⚠️ The exact numbers for *this* specific checkpoint may differ depending on training run and configuration. The values above are from the paper’s best configuration (RMC\*).

---

## 2. Intended Use

### What this model is for

- **Research on human–AI co-authorship**  
  - Studying where LLMs “take over” in mixed texts.  
  - Analysing robustness of detectors under adversarial perturbations.

- **Tooling / applications (with human oversight)**  
  - Assisting editors, educators, or moderators to **highlight suspicious spans** rather than making final decisions.  
  - Exploring **interpretability overlays** (e.g., heatmaps over tokens) when combined with Info-Mask outputs.

### What this model is *not* for

- Automated “cheating detector” / plagiarism court.  
- High-stakes decisions affecting people’s livelihood, grades, or reputation **without human review**.  
- Non-English or heavily code-mixed text (training data is English-centric). 

Use this model as a **signal**, not a judge.

---

## 3. Data: DAMASHA-MAS

The model is trained on the **MAS** benchmark released with the DAMASHA paper and hosted as the Hugging Face dataset:

- **Dataset:** [`saiteja33/DAMASHA`](https://huggingface.co/datasets/saiteja33/DAMASHA) 

### 3.1 What’s in MAS?

MAS consists of **mixed human–AI texts with explicit span tags**:  

- Human text comes from several corpora for **domain diversity**, including:  
  - Reddit (M4-Reddit)  
  - Yelp & /r/ChangeMyView (MAGE-YELP, MAGE-CMV)  
  - News summaries (XSUM)  
  - Wikipedia (M4-Wiki, MAGE-SQuAD)  
  - ArXiv abstracts (MAGE-SciGen)  
  - QA texts (MAGE-ELI5)

- AI text is generated by multiple modern LLMs:  
  - **DeepSeek-V3-671B** (open-source)  
  - **GPT-4o, GPT-4.1, GPT-4.1-mini** (closed-source)  

### 3.2 Span tagging

Authorship is marked using **explicit tags** around AI spans:  

- `<AI_Start>` … `</AI_End>` denote AI-generated segments within otherwise human text.  
- The dataset stores text in a `hybrid_text` column, plus metadata such as `has_pair`, and adversarial variants include `attack_name`, `tag_count`, and `attacked_text`.  
- Tags are sentence-level in annotation, but the model is trained to output **token-level** predictions for finer segmentation.

> During training, these tags are converted into **token labels** (2 labels total; see `config.id2label` in the model files).

### 3.3 Adversarial attacks

MAS includes multiple **syntactic attacks** applied to the mixed text:  

- Misspelling  
- Unicode character substitution  
- Invisible characters  
- Punctuation substitution  
- Upper/lower case swapping  
- All-mixed combinations of the above  

These perturbations make tokenization brittle and test robustness of detectors in realistic settings.

---

## 4. Model Architecture & Training

### 4.1 Architecture (conceptual)

The model follows the **Info-Mask RMC\*** architecture described in the DAMASHA paper:  

1. **Dual encoders**  
   - RoBERTa-base and ModernBERT-base encode the same input sequence.  
2. **Feature fusion**  
   - Hidden states from both encoders are fused into a shared representation.  
3. **Stylometric Info-Mask**  
   - Hand-crafted style features (perplexity, POS density, punctuation density, lexical diversity, readability) are projected, passed through multi-head attention, and turned into a **scalar mask per token**.  
   - This mask gates the fused encoder states, down-weighting style-irrelevant tokens and emphasizing style-diagnostic ones. :contentReference[oaicite:16]{index=16}  
4. **Sequence model + CRF**  
   - A BiGRU layer captures sequential dependencies, followed by a **CRF** layer for structured token labeling with a sequence-level loss. :contentReference[oaicite:17]{index=17}  

### 4.2 Training setup (from the paper)

Key hyperparameters used for the Info-Mask models on MAS:  

- **Number of labels:** 2  
- **Max sequence length:** 512  
- **Batch size:** 64  
- **Epochs:** 5  
- **Optimizer:** AdamW (with cosine annealing LR schedule)  
- **Weight decay:** 0.01  
- **Gradient clipping:** 1.0  
- **Dropout:** Dynamic 0.1–0.3 (initial 0.1)  
- **Warmup ratio:** 0.1  
- **Early stopping patience:** 2  

**Hardware & compute** (as reported): 

- AWS EC2 g6e.xlarge, NVIDIA L40S (48GB) GPU, Ubuntu 24.04  
- ≈ 400 GPU hours for experiments.

> The exact training script used for this checkpoint is available in the project GitHub:  
> <https://github.com/saitejalekkala33/DAMASHA>

---

---
license: mit
---