BananaMind/BananaMind-2-AI-Detect

BananaMind 2 AI Detect

Demo: https://huggingface.co/spaces/Banaxi-Tech/BananaMind-2-AI-Detect-Demo

BananaMind 2 AI Detect is a human-vs-AI text detector built by fully fine-tuning Qwen/Qwen3.5-0.8B-Base on the RAID benchmark. It is our first dedicated AI-text detection model!

The model is a base-model-style completion classifier: given a passage, it completes Label: with either human or ai, and the human/ai token logits at that position give a calibrated confidence score.

Model Details

Field	Value
Base model	Qwen/Qwen3.5-0.8B-Base
Parameters	0.8B
Fine-tune type	Full fine-tune (all weights)
Architecture	Qwen3.5 hybrid Gated DeltaNet + attention, Qwen3_5ForCausalLM
Vocab size	248,320
Context length (base)	262,144
Max sequence length (training)	2,048
Task format	Completion: Text: <passage>\nLabel: human / Text: <passage>\nLabel: ai
Weight format	safetensors

Training Data

Trained on the RAID benchmark (ACL 2024), restricted to two classes: human and gpt4.

Class	Examples
Human	150,000
GPT-4	150,000
Total prepared	300,000
Train / Eval split (prepared)	294,000 / 6,000
Train examples actually seen	57,600 (20% of the prepared train set, due to compute limits — see Max steps below)

RAID is licensed MIT. This restricts training to a single AI source (GPT-4) rather than the full 11-generator set — see Limitations below for what that means in practice.

Training Setup

Field	Value
Hardware	1x RTX PRO 6000 Blackwell (96GB)
Fine-tune type	Full fine-tune, no LoRA
Precision	bf16
Sequence length	2,048
Micro batch	12
Gradient accumulation	4
Effective batch	48 sequences
Max steps	1,200 — capped due to compute limits (≈57,600 examples seen, ~20% of one epoch over the 294,000-example train set)
Optimizer	AdamW
Peak learning rate	1e-5
LR schedule	Cosine, warmup ratio 0.03
Weight decay	0.01
Gradient checkpointing	Off
Kernels	flash-linear-attention, causal-conv1d (fused Gated DeltaNet path)

Evaluation

Self-reported. Label accuracy measured by comparing the argmax of the human / ai logits against ground truth on held-out RAID examples.

Eval	Result
In-distribution label accuracy (RAID gpt4/human held-out set, n=200)	99.5%
Eval loss @ step 300	1.82
Eval mean token accuracy @ step 300	60.1%

Manual spot checks (single examples, not a formal benchmark):

Input	Prediction	P(ai)
GPT-4 poem, unedited	ai	99%
Human-written poem	human	2%
Gemini poem	ai	90%
Claude poem	human	1.2%
GPT-4 self-edited poem	ai	96%
GPT-4 poem, prompted to "sound human"	ai	96%

Limitations

This model was trained for 1,200 steps (~57,600 examples, roughly 20% of one epoch over the 294,000-example train set) rather than to convergence over the full dataset — the run was deliberately capped to fit a time budget. Loss and eval-loss were still trending downward at the final step, so further training on the same data would likely improve results further, particularly on harder or out-of-distribution cases.

This model was also trained on a single AI generator (GPT-4) versus human text, not RAID's full 11-model roster. In practice this means:

• It is closer to a "GPT-4 vs. not-GPT-4" classifier than a general AI-text detector. Text from other AI systems (Claude, Gemini, Llama, Mistral, etc.) was never shown as a positive "ai" example during training, so "human" predictions on non-GPT-4 AI text should not be read as verified human authorship.
• It has not been evaluated against paraphrasing or other adversarial rewriting. RAID's own adversarial-attack subset (paraphrase, synonym, article_deletion, etc.) was not included in training or in the spot checks below, so robustness to text that has been reworded or lightly edited is untested.
• It is comparatively robust to GPT-4 self-editing and "sound human" prompting — both stayed correctly classified as ai in spot testing, suggesting it picked up signal beyond surface phrasing for at least the GPT-4 case.
• Evaluation numbers above are drawn from the same RAID distribution used for training (in-distribution), plus a handful of manual examples — not a comprehensive out-of-distribution benchmark.

Usage

pip install -U transformers torch

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "BananaMind/BananaMind-2-AI-Detect"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

HUMAN_ID = tokenizer.encode(" human", add_special_tokens=False)[0]
AI_ID = tokenizer.encode(" ai", add_special_tokens=False)[0]

def classify(text: str, max_chars: int = 6000):
    prompt = f"Text: {text[:max_chars]}\nLabel:"
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        logits = model(**inputs).logits[0, -1]
    probs = torch.softmax(logits[[HUMAN_ID, AI_ID]].float(), dim=0)
    label = "human" if probs[0] > probs[1] else "ai"
    return label, probs[1].item()  # (prediction, P(ai))

label, p_ai = classify("Your text here.")
print(label, p_ai)

Intended Use

BananaMind 2 AI Detect is intended for lightweight, local experimentation with AI-text detection, research into detector robustness, and as a baseline for comparison against RAID's leaderboard detectors. It is not intended as a sole or authoritative source for high-stakes decisions (academic integrity, content moderation, legal contexts) given the single-generator training scope and known paraphrase fragility described above.

License

Apache 2.0 (inherited from the base model, Qwen/Qwen3.5-0.8B-Base). Training data (RAID) is MIT-licensed; underlying human-authored source texts retain their original third-party rights.

Citation

If you use RAID, please cite the original paper:

@inproceedings{dugan-etal-2024-raid,
    title = "{RAID}: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors",
    author = "Dugan, Liam  and
      Hwang, Alyssa  and
      Trhl{\'\i}k, Filip  and
      Zhu, Andrew  and
      Ludan, Josh Magnus  and
      Xu, Hainiu  and
      Ippolito, Daphne  and
      Callison-Burch, Chris",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.acl-long.674",
    pages = "12463--12492",
}

🍌