SallySims commited on
Commit
02d73ae
Β·
verified Β·
1 Parent(s): de49970

Add detailed model card

Browse files
Files changed (1) hide show
  1. README.md +148 -0
README.md ADDED
@@ -0,0 +1,148 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: apache-2.0
4
+ tags:
5
+ - pytorch
6
+ - text-classification
7
+ - dei
8
+ - bias-detection
9
+ - equibert
10
+ datasets:
11
+ - synthetic
12
+ metrics:
13
+ - f1
14
+ - roc_auc
15
+ ---
16
+
17
+ # EquiBERT β€” Bias Classifier
18
+
19
+ **Model ID:** `SallySims/equibert-bias-classifier`
20
+
21
+ Multi-label classifier that detects seven types of bias in
22
+ organisational text β€” job descriptions, HR communications,
23
+ policies, and workplace language.
24
+
25
+ ## Labels
26
+
27
+ | ID | Label | Description |
28
+ |----|-------|-------------|
29
+ | 0 | `gender_bias` | Gendered language, role assumptions, masculine-coded words |
30
+ | 1 | `racial_bias` | Racial coding, cultural fit language, tokenism |
31
+ | 2 | `age_bias` | Digital native language, overqualified framing, generational stereotypes |
32
+ | 3 | `ability_bias` | Ableist language, physical requirements, disability framing |
33
+ | 4 | `socioeconomic_bias` | Class-coded language, credential gatekeeping |
34
+ | 5 | `cultural_bias` | Cultural exclusion, religious insensitivity |
35
+ | 6 | `intersectional` | Compounding bias across multiple identity dimensions |
36
+
37
+ ## Usage
38
+
39
+ ```python
40
+ from transformers import AutoTokenizer
41
+ import torch
42
+
43
+ tokenizer = AutoTokenizer.from_pretrained("SallySims/equibert-bias-classifier")
44
+
45
+ text = "We need a rock star developer who can dominate the roadmap."
46
+ inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
47
+
48
+ # Load model weights and run inference
49
+ # (use with the EquiBERT modeling code from the repository)
50
+ ```
51
+
52
+ ## Task Head Architecture
53
+
54
+ ```
55
+ CLS token β†’ Dropout(0.1) β†’ Linear(hidden, hidden//2) β†’ GELU β†’ Linear(hidden//2, 7)
56
+ ↓
57
+ BCEWithLogitsLoss (multi-label)
58
+ Sigmoid threshold @ 0.5
59
+ ```
60
+
61
+ ## Performance (synthetic data, seed=42)
62
+
63
+ | Metric | Score |
64
+ |--------|-------|
65
+ | Macro F1 | 0.72 |
66
+ | Micro F1 | 0.76 |
67
+ | AUC | 0.81 |
68
+
69
+ ## Model Description
70
+
71
+ EquiBERT is a multi-task DEI (Diversity, Equity and Inclusion) transformer
72
+ built on a dual-encoder backbone that fuses **RoBERTa-base** and
73
+ **DeBERTa-v3-base** via a learned weighted sum (Ξ± parameter).
74
+ The fused representation is fed into task-specific heads covering
75
+ 17 distinct DEI analysis tasks.
76
+
77
+ **Organisation:** [SallySims](https://huggingface.co/SallySims)
78
+ **Framework:** PyTorch + HuggingFace Transformers
79
+ **Backbone:** RoBERTa-base + DeBERTa-v3-base (dual encoder, fused)
80
+ **Language:** English
81
+ **Domain:** Organisational DEI text β€” HR communications, policies,
82
+ job descriptions, performance reviews, leadership statements, reports
83
+
84
+ ## Architecture
85
+
86
+ ```
87
+ Input Text
88
+ β”‚
89
+ β”œβ”€β”€β–Ά RoBERTa-base encoder ──▢ Linear projection
90
+ β”‚ β”‚
91
+ └──▢ DeBERTa-v3-base encoder ──▢ Linear projection
92
+ β”‚
93
+ Weighted fusion (learned Ξ±)
94
+ β”‚
95
+ Layer Norm + Dropout
96
+ β”‚
97
+ Task-specific head (see below)
98
+ ```
99
+
100
+ ## Training Data
101
+
102
+ Trained on synthetic DEI organisational text generated by the
103
+ EquiBERT synthetic data pipeline, covering 20 DEI categories
104
+ across HR, policy, leadership, and workforce analytics domains.
105
+ For production use, fine-tune on real labelled DEI data.
106
+
107
+ ## Limitations
108
+
109
+ - Trained on synthetic data β€” predictions should be validated
110
+ before use in real HR or policy decisions.
111
+ - English-only.
112
+ - Not a substitute for qualified DEI practitioners or legal advice.
113
+ - May reflect biases present in the training corpus.
114
+
115
+ ## Citation
116
+
117
+ If you use EquiBERT in your research, please cite:
118
+
119
+ ```bibtex
120
+ @misc{equibert2024,
121
+ author = {SallySims},
122
+ title = {EquiBERT: A Multi-Task DEI Transformer},
123
+ year = {2024},
124
+ publisher = {HuggingFace},
125
+ url = {https://huggingface.co/SallySims}
126
+ }
127
+ ```
128
+
129
+ ## All EquiBERT Models
130
+
131
+ | Model | Task | Primary Metric |
132
+ |-------|------|---------------|
133
+ | [equibert-bias-classifier](https://huggingface.co/SallySims/equibert-bias-classifier) | Bias Detection | Macro F1 |
134
+ | [equibert-microaggression](https://huggingface.co/SallySims/equibert-microaggression) | Microaggression Detection | Macro F1 |
135
+ | [equibert-category-tagger](https://huggingface.co/SallySims/equibert-category-tagger) | DEI Category Tagging | Macro F1 |
136
+ | [equibert-event-exclusion](https://huggingface.co/SallySims/equibert-event-exclusion) | Event Exclusion Classification | Macro F1 |
137
+ | [equibert-inclusive-language](https://huggingface.co/SallySims/equibert-inclusive-language) | Inclusive Language Scoring | Span F1 |
138
+ | [equibert-review-auditor](https://huggingface.co/SallySims/equibert-review-auditor) | Performance Review Auditing | Span F1 |
139
+ | [equibert-washing-detector](https://huggingface.co/SallySims/equibert-washing-detector) | DEI Washing Detection | MAE |
140
+ | [equibert-framing-scorer](https://huggingface.co/SallySims/equibert-framing-scorer) | Report Framing Scoring | MAE |
141
+ | [equibert-awareness-scorer](https://huggingface.co/SallySims/equibert-awareness-scorer) | DEI Awareness Scoring | MAE |
142
+ | [equibert-similarity](https://huggingface.co/SallySims/equibert-similarity) | Semantic Similarity | Accuracy |
143
+ | [equibert-ner](https://huggingface.co/SallySims/equibert-ner) | DEI Entity Recognition | Span F1 |
144
+ | [equibert-relation-extraction](https://huggingface.co/SallySims/equibert-relation-extraction) | Relation Extraction | Macro F1 |
145
+ | [equibert-qa](https://huggingface.co/SallySims/equibert-qa) | Extractive QA | Span EM |
146
+ | [equibert-search](https://huggingface.co/SallySims/equibert-search) | Semantic Search | MRR@10 |
147
+ | [equibert-nli](https://huggingface.co/SallySims/equibert-nli) | NLI / Textual Entailment | Macro F1 |
148
+ | [equibert-generator](https://huggingface.co/SallySims/equibert-generator) | DEI Text Generation | ROUGE-L |