File size: 879 Bytes
9302284 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | # Error Analysis (Privacy-Preserving)
The assignment expects an error analysis section explaining what the model fails on and why.
To avoid printing or storing offensive content, we implement error analysis that:
- does **not** write raw text
- only outputs aggregate statistics + hashed identifiers (`sha256(text)`)
## What we compute
- overall metrics at a threshold
- confusion counts per label (TP/FP/FN/TN)
- feature summaries for FP vs FN:
- length (chars/words)
- uppercase ratio
- punctuation ratio
- URL/email presence
- repeated characters
- non-ascii ratio
- top error cases (hashed only) with:
- per-label probabilities and true/pred labels
- numeric features
## How to run
```bash
toxicity-agent error-analysis --config configs/train.yaml --split test --threshold 0.5
```
Output: `artifacts/runs/error_analysis/error-analysis-<timestamp>.json`
|