BillFisk/ai-text-detector

Clueso AI-text detector, LoRA v2

The model behind the @bandf/clueso npm package: a DeBERTa-v3-large classifier with mask-aware mean pooling and four output classes (human, ai, ai_edited, humanized), fine-tuned with rank-8 LoRA adapters merged into the weights and exported to ONNX. The headline score is 1 − P(human).

Files

File	Bytes	SHA-256
onnx/model.onnx (fp32)	1,737,980,451	74b9fcfbf25eab67bb963818643790e2556ffb548afd5752149da1f15db54877
onnx/model_fp16.onnx	870,662,236	fc9d24662bf76b4a8084445365991eba9a77aeb0dc56d97c747ae658ed86237b

The fp16 file keeps activations in fp32 and yields the same decisions as fp32: on 600 evaluation documents log-odds differ by at most 0.04 and no verdict changes. It is about half again slower on CPU. manifest.json pins both variants for the npm installer; config.json, tokenizer.json, and tokenizer_config.json load with Transformers.js.

Use

Through the package, which adds input normalization, sentence-aligned chunking, evidence windows, and calibrated modes:

npm install @bandf/clueso

import { detect } from '@bandf/clueso';
const result = await detect(text);
console.log(result.verdict, result.margin);

Directly with Transformers.js, scoring one input of up to 768 tokens:

import { AutoModelForSequenceClassification, AutoTokenizer } from '@huggingface/transformers';

const id = 'BillFisk/ai-text-detector';
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await AutoModelForSequenceClassification.from_pretrained(id, { dtype: 'fp32' });
const encoded = await tokenizer(text, { truncation: true, max_length: 768 });
const logits = (await model(encoded)).logits.tolist()[0];
const max = Math.max(...logits);
const probabilities = logits.map((v) => Math.exp(v - max)).map((v, _, all) => v / all.reduce((s, x) => s + x, 0));
const aiScore = 1 - probabilities[0];

Operating thresholds

Calibrated on a 3,861-document partition that includes essays by English language learners. Each mode has a document threshold and a threshold for the highest 128-token evidence window, chosen jointly so that the share of human documents receiving any flag matches the target.

Mode	Target human FPR	Document threshold	Window threshold
strict	0.5%	0.9990641518953358	0.9999630607884616
balanced	2%	0.996158481097896	0.9991614647843164
sensitive	5%	0.9782890757665377	0.9973714937509289

Results

Sealed evaluation, 2,161 documents never used for selection or calibration, strict mode, compared with the previous release (v1, MAGE-only fine-tune) under the same decision rules:

Sealed slice	v1	v2
AI from current models (llama3.3 70B, qwen3.8 27B, gemma4 26B, glm-4.7-flash)	81.3%	95.4%
AI rewritten to sound human	61.0%	93.2%
RAID paraphrase attack	43.2%	63.6%
RAID, all eleven attacks	67.4%	79.0%
AI texts of 50–99 words	44.4%	66.7%
Essays by English language learners wrongly flagged	6.8%	0.2%
Human documents receiving any flag	0.0–2.7%	0.0%

llama3.3 and qwen3.8 were held out of training. Full reports are in the package repository under evaluation/results/.

Training data

Source	Human	AI
MAGE training split (Apache-2.0)	4,000	3,999
Cambridge Write & Improve + LOCNESS, FCE (non-commercial research licences), grouped by writer	3,000	–
RAID training split (MIT): clean, paraphrased, and synonym-substituted human and AI text	2,297	3,166
Local generations: gemma4 26B, glm-4.7-flash, gpt-oss, granite4.1 8B, mistral 7B, nemotron-3-nano 30B, qwen3 30B-A3B, at temperatures 0.3–1.0, plus humanized rewrites	8	2,885

Training rows are the package's own 768-token chunks plus one sentence-aligned 96–256-token window per document. One epoch, LoRA rank 8 on query and value projections, 790,532 trainable parameters, three seeds; this is seed 3, chosen on validation partial ROC-AUC below 1% false-positive rate (0.765; ROC-AUC 0.988). PyTorch–ONNX parity: maximum logit difference 2.19e-5.

Write & Improve, LOCNESS, and FCE are licensed for non-commercial research and educational use and exclude information derived from them in commercial products. These weights are published as part of a non-commercial open-source package on that basis. Citations are in NOTICE.

Limitations

• Paraphrased AI text: 64% recall in strict mode.
• Short, formulaic test essays (TOEFL): 3 of 91 flagged in strict mode.
• English only. Minimum 50 words.
• Generators newer than the training data will drift.
• Detector output is evidence, never proof of authorship. Do not use it as the sole basis for disciplinary or other high-impact decisions.