| --- |
| license: cc-by-nc-nd-4.0 |
| datasets: |
| - AdamCodd/Civitai-15M-prompts |
| language: |
| - en |
| metrics: |
| - precision |
| - accuracy |
| - roc_auc |
| - recall |
| - f1 |
| base_model: |
| - jhu-clsp/ettin-encoder-68m |
| tags: |
| - text-classification |
| - nsfw-detection |
| - stable-diffusion |
| - ettin |
| - modernbert |
| - safety |
| - transformers |
| - nlp |
| model-index: |
| - name: ettin-nsfw-prompt-stable-diffusion |
| results: |
| - task: |
| type: text-classification |
| name: Text Classification |
| metrics: |
| - type: accuracy |
| value: 0.9317 |
| name: Accuracy |
| - type: f1 |
| value: 0.9308 |
| name: F1 |
| - type: precision |
| value: 0.9430 |
| name: Precision |
| - type: recall |
| value: 0.9189 |
| name: Recall |
| - type: ROC_AUC |
| value: 0.9823 |
| name: AUC |
| extra_gated_prompt: >- |
| To request commercial use rights, API access, or enterprise deployment, please email adamcoddml@gmail.com with a brief overview of your organization and intended use case. Unverified or incomplete requests cannot be processed. |
| extra_gated_fields: |
| Company/University: text |
| Country: country |
| --- |
| |
| # Ettin-nsfw-prompt-stable-diffusion |
|
|
| A highly efficient, optimized text classifier designed to detect NSFW prompts in Stable Diffusion and other text-to-image pipelines. Built with an 8,192-token context window and a lightweight 68M parameter architecture, it delivers faster inference and tighter safety filtering than legacy BERT-based models. |
|
|
| **Commercial Use & Licensing:** This model is gated under a **cc-by-nc-nd-4.0** license. Please see the prompt above to request commercial use rights or API access. |
|
|
|
|
| ## Evaluation Metrics |
|
|
| Evaluated on a held-out test set of 100,000 samples. |
|
|
| | Metric | Ettin-68M (This Model) | DistilRoBERTa V3 (Previous) | |
| | :--- | :--- | :--- | |
| | **ROC-AUC** | **0.9823** | 0.9764 | |
| | **Accuracy** | **0.9317** | 0.9198 | |
| | **F1 Score** | **0.9308** | 0.9201 | |
| | **Precision** | **0.9430** | 0.9179 | |
| | **Recall** | 0.9189 | **0.9222** | |
|
|
| *(Note: While DistilRoBERTa shows a slightly higher base recall precisely at the 0.5 threshold, Ettin-68M scales far better across the curve, evidenced by its superior ROC-AUC).* |
|
|
|
|
| ## <u>TL;DR</u> |
|
|
| * **Long Context Window (8,192 tokens)**: A 16x increase over legacy RoBERTa models. Catch malicious keywords hidden at the very end of heavy negative prompts without sequence truncation. |
|
|
| * **Lighter & Faster (68M Params)**: A 17% reduction in size compared to DistilRoBERTa, enabling higher API throughput and lower VRAM consumption for real-time edge filtering. |
|
|
| * **Production-Calibrated Safety**: Pre-mapped operational thresholds (e.g., 0.0487) to guarantee a strict 99.0% recall rate in enterprise environments. |
|
|
| * **Modern Foundations**: Built on the JHU-CLSP Ettin architecture, using RoPE and multi-stage context extension. |
|
|
| ## Overview |
|
|
| Trained on [14.9 million strictly balanced Stable Diffusion-style prompts](https://huggingface.co/datasets/AdamCodd/Civitai-15M-prompts) (with 100K reserved for testing), this model is a strict performance upgrade over our previous [DistilRoBERTa classifier](https://huggingface.co/AdamCodd/distilroberta-nsfw-prompt-stable-diffusion). |
|
|
| By replacing legacy BERT-era foundations with Johns Hopkins University’s `jhu-clsp/ettin-encoder-68m` [[1]](#ettin) a state-of-the-art 2025 open-data replication of ModernBERT, we unlock significant modern transformer advancements that directly improve production moderation. |
|
|
|
|
| ## Results on Production Usage & Safety Thresholds |
|
|
| By default, standard Hugging Face pipelines use a default classification threshold of `0.5`. However, for production pipelines prioritizing safety (e.g., enterprise APIs or public web UIs), thresholds must be calibrated to hit specific Recall targets. |
|
|
| We ran extensive PR-AUC (Precision-Recall Area Under Curve) evaluations on our 100K test set to map the exact operational threshold required to hit a **99.0% Recall** target (ensuring the model catches 99% of all actual NSFW prompts). |
|
|
| * **Overall PR-AUC:** 0.9844 |
| * **Optimal Safety Threshold:** `0.0487` |
| * **Expected Recall:** 99.00% |
| * **Expected Precision:** 74.80% |
|
|
| To implement this strict safety filter, configure your pipeline to flag a prompt if the positive NSFW probability exceeds `0.0487` rather than the default `0.5`. This rigorous threshold guarantees 99% SFW enforcement, with a manageable ~25% false-positive rate as a trade-off for commercial safety. |
|
|
|
|
| ## Usage |
|
|
| ```python |
| from transformers import pipeline |
| |
| classifier = pipeline("text-classification", model="AdamCodd/ettin-nsfw-prompt-stable-diffusion", top_k=None) |
| |
| # Predict |
| result = classifier("A beautiful landscape painting, mountains, 8k resolution") |
| |
| # Enforce rigorous commercial safety threshold (99.0% Recall) |
| nsfw_prob = next(score['score'] for score in result[0] if score['label'] == 'NSFW') |
| is_flagged = nsfw_prob > 0.0487 |
| ``` |
|
|
| ## Training Details |
| * **Base Model:** `jhu-clsp/ettin-encoder-68m` |
| * **Dataset:** 14.9 million Stable Diffusion-style prompts (balanced exactly 50/50 between NSFW and SFW). |
| * **Test Set:** 100,000 prompts. |
|
|
| --- |
| ### References |
| * <a id="ettin"></a>**[1]** Weller et al., 2025. ["Ettin: Analyzing Encoders vs Decoders Using the Same Architecture and Data"](https://arxiv.org/abs/2507.11412). arXiv:2507.11412. |
|
|
| * <a id="bert"></a>**[2]** Hugging Face Documentation. DistilRoBERTa Base Model Specifications (82M Parameters). |
