README.md

---
language: en
license: apache-2.0
tags:
  - pytorch
  - text2text-generation
  - dei
  - text-generation
  - t5
  - equibert
metrics:
  - rouge
  - bleu
---

# EquiBERT — DEI Text Generator

**Model ID:** `SallySims/equibert-generator`

T5-base fine-tuned for conditional DEI text generation.
Generates inclusive, equitable organisational text across
seven task types given a task prefix and input.

## Task Prefixes

| Prefix | Task | Input → Output |
|--------|------|----------------|
| `rewrite inclusive:` | Inclusive rewriting | Biased text → Inclusive version |
| `generate policy:` | Policy generation | Topic → Policy section |
| `generate jd:` | Job description | Role description → Inclusive JD |
| `rewrite framing:` | Framing correction | Victim-blaming text → Structural framing |
| `generate commitment:` | DEI commitment | Goal → Measurable commitment |
| `rewrite review:` | Review debiasing | Biased review → Unbiased version |
| `generate awareness:` | Awareness content | Topic → Awareness statement |

## Usage

```python
from transformers import T5ForConditionalGeneration, T5Tokenizer

model     = T5ForConditionalGeneration.from_pretrained("SallySims/equibert-generator")
tokenizer = T5Tokenizer.from_pretrained("SallySims/equibert-generator")

prompt = "rewrite inclusive: We need a rock star developer who can dominate the roadmap."
inputs = tokenizer(prompt, return_tensors="pt", max_length=256, truncation=True)
output = model.generate(**inputs, max_new_tokens=200, num_beams=4)
result = tokenizer.decode(output[0], skip_special_tokens=True)
print(result)
# "We are looking for a skilled developer with strong technical expertise
#  who can contribute meaningfully to our product roadmap."
```

## Applications

- Automated inclusive job description generation
- DEI report framing improvement
- Performance review debiasing assistance
- Policy language generation
- Leadership communication coaching

## Model Description

EquiBERT is a multi-task DEI (Diversity, Equity and Inclusion) transformer
built on a dual-encoder backbone that fuses **RoBERTa-base** and
**DeBERTa-v3-base** via a learned weighted sum (α parameter).
The fused representation is fed into task-specific heads covering
17 distinct DEI analysis tasks.

**Organisation:** [SallySims](https://huggingface.co/SallySims)
**Framework:** PyTorch + HuggingFace Transformers
**Backbone:** RoBERTa-base + DeBERTa-v3-base (dual encoder, fused)
**Language:** English
**Domain:** Organisational DEI text — HR communications, policies,
job descriptions, performance reviews, leadership statements, reports

## Architecture

```
Input Text
    │
    ├──▶ RoBERTa-base encoder ──▶ Linear projection
    │                                     │
    └──▶ DeBERTa-v3-base encoder ──▶ Linear projection
                                          │
                              Weighted fusion (learned α)
                                          │
                                   Layer Norm + Dropout
                                          │
                              Task-specific head (see below)
```

## Training Data

Trained on synthetic DEI organisational text generated by the
EquiBERT synthetic data pipeline, covering 20 DEI categories
across HR, policy, leadership, and workforce analytics domains.
For production use, fine-tune on real labelled DEI data.

## Limitations

- Trained on synthetic data — predictions should be validated
  before use in real HR or policy decisions.
- English-only.
- Not a substitute for qualified DEI practitioners or legal advice.
- May reflect biases present in the training corpus.

## Citation

If you use EquiBERT in your research, please cite:

```bibtex
@misc{equibert2024,
  author    = {SallySims},
  title     = {EquiBERT: A Multi-Task DEI Transformer},
  year      = {2024},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/SallySims}
}
```

## All EquiBERT Models

| Model | Task | Primary Metric |
|-------|------|---------------|
| [equibert-bias-classifier](https://huggingface.co/SallySims/equibert-bias-classifier) | Bias Detection | Macro F1 |
| [equibert-microaggression](https://huggingface.co/SallySims/equibert-microaggression) | Microaggression Detection | Macro F1 |
| [equibert-category-tagger](https://huggingface.co/SallySims/equibert-category-tagger) | DEI Category Tagging | Macro F1 |
| [equibert-event-exclusion](https://huggingface.co/SallySims/equibert-event-exclusion) | Event Exclusion Classification | Macro F1 |
| [equibert-inclusive-language](https://huggingface.co/SallySims/equibert-inclusive-language) | Inclusive Language Scoring | Span F1 |
| [equibert-review-auditor](https://huggingface.co/SallySims/equibert-review-auditor) | Performance Review Auditing | Span F1 |
| [equibert-washing-detector](https://huggingface.co/SallySims/equibert-washing-detector) | DEI Washing Detection | MAE |
| [equibert-framing-scorer](https://huggingface.co/SallySims/equibert-framing-scorer) | Report Framing Scoring | MAE |
| [equibert-awareness-scorer](https://huggingface.co/SallySims/equibert-awareness-scorer) | DEI Awareness Scoring | MAE |
| [equibert-similarity](https://huggingface.co/SallySims/equibert-similarity) | Semantic Similarity | Accuracy |
| [equibert-ner](https://huggingface.co/SallySims/equibert-ner) | DEI Entity Recognition | Span F1 |
| [equibert-relation-extraction](https://huggingface.co/SallySims/equibert-relation-extraction) | Relation Extraction | Macro F1 |
| [equibert-qa](https://huggingface.co/SallySims/equibert-qa) | Extractive QA | Span EM |
| [equibert-search](https://huggingface.co/SallySims/equibert-search) | Semantic Search | MRR@10 |
| [equibert-nli](https://huggingface.co/SallySims/equibert-nli) | NLI / Textual Entailment | Macro F1 |
| [equibert-generator](https://huggingface.co/SallySims/equibert-generator) | DEI Text Generation | ROUGE-L |