Josephaimodels/manipulation-flag-models

These two classifier models are fine-tuned to flag possible manipulation in messages, having been trained on synthetic interpersonal relationship data.

The smaller model is based on microsoft/xtremedistil-l6-h256-uncased and has 12.75M total parameters. The larger uses microsoft/deberta-v3-xsmall and is at 70.83M total parameters. Both models achieve +99% F1 score on the held out test split.

The confidence score of the predictions are scaled to reflect the probability of the prediction being true, however there are instances when the models predict a blatantly wrong answer with full confidence. Furthermore, if the message requires additional context to be manipulative, then it is considered bening.

The training data was augmented to make the models robust to typos and adversarial attacks, but highest accuracy is achieved on clean text.

Inference scripts are provided alongside the models for quick setup.

Both models are released under the MIT license.