Text Classification
Transformers
PyTorch
English
distilbert
toxic text classification
text-embeddings-inference
Instructions to use tensor-trek/distilbert-toxicity-classifier with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tensor-trek/distilbert-toxicity-classifier with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="tensor-trek/distilbert-toxicity-classifier")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("tensor-trek/distilbert-toxicity-classifier") model = AutoModelForSequenceClassification.from_pretrained("tensor-trek/distilbert-toxicity-classifier", device_map="auto") - Notebooks
- Google Colab
- Kaggle
metadata
language:
- en
tags:
- toxic text classification
licenses:
- apache-2.0
Toxicity Classification Model
This model is trained for toxicity classification task using. The dataset used for training is the dataset by Jigsaw ( Jigsaw 2020). We split it into two parts and fine-tune a DistilBERT model (DistilBERT base model (uncased) ) on it. DistilBERT is a distilled version of the BERT base model. It was introduced in this paper.
How to use
from transformers import pipeline
text = "This was a masterpiece. Not completely faithful to the books, but enthralling from beginning to end. Might be my favorite of the three."
classifier = pipeline("text-classification", model="tensor-trek/distilbert-toxicity-classifier")
classifier(text)