---
title: PrismAudit | Responsible ML Evaluation Studio
emoji: 🔎
colorFrom: green
colorTo: blue
sdk: static
app_file: index.html
pinned: false
license: mit
short_description: Local-first classifier evaluation and fairness analysis
tags:
  - machine-learning
  - model-evaluation
  - classification
  - responsible-ai
  - explainability
  - fairness
  - data-analysis
  - javascript
---

# PrismAudit | Responsible ML Evaluation Studio

**Project 24 of the Hugdrigo portfolio** · An open-source, local-first studio for evaluating binary classification scores, understanding decision thresholds, checking probability calibration and measuring differences across user-defined cohorts.

[**Open the Space**](https://huggingface.co/spaces/Hugdrigo/prismaudit-model-lab)

![License: MIT](https://img.shields.io/badge/License-MIT-informational?style=flat-square) ![Platform: Static Space](https://img.shields.io/badge/Platform-Static%20Space-informational?style=flat-square) ![Backend: None](https://img.shields.io/badge/Backend-None-success?style=flat-square) ![Project: 24](https://img.shields.io/badge/Portfolio-Project%2024-93dc85?style=flat-square)

> **Interpretation limits:** PrismAudit **does not train a machine-learning model**, predict new scores, prove fairness or detect unlawful discrimination. It evaluates *existing* binary scores against known labels. Built-in sample data are **fictional and synthetic**, not measurements of any real company, people or deployed model. Reports are descriptive and do not constitute validation for deployment or legal compliance.

## Overview

Many classification dashboards stop at accuracy, hiding important trade-offs and subgroup differences. PrismAudit turns an existing set of probability scores into an interpretable, reproducible evaluation report that runs entirely in a browser.

| Module | Functions |
| --- | --- |
| **Overview** | Accuracy, F1, ROC AUC, Brier score; confusion matrix, ROC plot, 10-bin calibration diagnostic and explainable audit signals. |
| **Threshold explorer** | Interactive decision cutoff (0.00–1.00); precision, recall, specificity and balanced accuracy; 21-point sensitivity sweep; highest *in-sample* F1 operating point; seeded bootstrap interval. |
| **Fairness lens** | Cohort decision rates, prevalence, recall, false positive rates and absolute max–min differences, with sample-size and missing-class warnings. |
| **Data & methods** | Local CSV import and schema validation, dataset profiling, synthetic example download and transparent methodology. |
| **Export** | Reproducible JSON audit with all aggregate metrics; CSV operating-point and cohort tables. |

### No paid infrastructure

- Runs as an ordinary **Hugging Face Static Space** (`sdk: static`).
- Uses **vanilla HTML, CSS and ECMAScript modules**. No framework, API key, API calls, build pipeline, npm packages or cloud processing.
- CSV processing, metric computation, chart rendering and export happen in browser memory.
- No analytics, CDN, trackers, external telemetry or network requests are initiated by the application itself. Hosting-provider logging may still occur when accessing the Space.

## Start here

1. Open the **App** tab in Hugging Face.
2. Explore a fictional sample: **Support prioritization**, **Quality review** or **Training completion**.
3. Move the threshold slider and observe the changing confusion matrix, F1, precision, recall and group metrics. AUC, calibration and Brier score remain threshold-independent.
4. Use **Threshold explorer** to view all operating points and a seeded bootstrap interval.
5. Use **Fairness lens** for disaggregated cohort rates. Treat differences as prompts for investigation, not findings of unfairness.
6. Under **Data & methods**, import your own CSV, or download the example CSV to see the expected format.
7. Export a JSON audit and selected CSV tables with dedicated buttons.

## CSV contract

Required columns (case-insensitive header names):

| Column | Example | Definition |
| --- | --- | --- |
| `score` | `0.86` | Numeric model-assigned score in the closed interval `[0, 1]`. Interpreted as an estimated positive-class probability for calibration analysis. |
| `label` | `1` | Known ground-truth binary outcome: `0` or `1`. |
| `group` | `North` | Categorical cohort assignment, nonempty, maximum 72 characters. |
| `id` | `CASE-001` | Optional row identifier; not required for any metric. |

Example:

```csv
id,score,label,group
CASE-001,0.86,1,North
CASE-002,0.22,0,South
CASE-003,0.62,0,North
CASE-004,0.14,1,Central
```

Requirements and behavior:

- File extension `.csv`, maximum **2 MiB**; at most **10,000 data rows** and **35 distinct cohorts**.
- Supports UTF-8 CSV files with `,` delimiters, CRLF line endings, quoted fields, doubled quotes and embedded line breaks in quoted fields.
- Rejects missing or duplicate required headers, missing groups, non-binary labels, invalid scores and malformed row lengths with actionable messages.
- No scores are produced, adjusted or retrained. The slider changes only the positive/negative *decision* cutoff.
- CSV rows are not included in JSON audit exports. Aggregate metrics and the imported **file name** may be present in the report, so review outputs before sharing.
- Samples with very few rows or only one label class can yield undefined metrics. The UI uses `—` and JSON uses `null`, **not zero**, in these cases.

### Dataset privacy

Imported data are parsed in the browser. PrismAudit's own code does not upload the file to any server. It does not persist data in cookies or browser storage. The application is not a substitute for an organization's data governance controls: prefer sanitized or synthetic datasets when evaluating potentially personal, regulated or confidential data.

## Methods and equations

Let **TP**, **FP**, **TN**, **FN** be the confusion counts at a chosen threshold `t`, with predicted positive iff `score >= t`.

| Measure | Definition |
| --- | --- |
| Accuracy | `(TP + TN) / N` |
| Precision | `TP / (TP + FP)` |
| Recall / true positive rate | `TP / (TP + FN)` |
| Specificity / true negative rate | `TN / (TN + FP)` |
| False positive rate | `FP / (FP + TN)` |
| F1 | `2 * precision * recall / (precision + recall)` |
| Balanced accuracy | `(recall + specificity) / 2` |
| Positive prediction rate | `(TP + FP) / N` |
| Brier score | `mean((score - label)^2)` |
| ECE | `sum_b((n_b / N) * abs(mean_score_b - observed_positive_rate_b))` for 10 equal-width score bins |
| ROC AUC | Tie-aware Mann–Whitney rank statistic, equivalent to ROC AUC for these empirical binary-score samples |

Division with a zero denominator yields an undefined rate. An F1 of zero is returned for defined zero-valued precision or recall; otherwise undefined inputs remain undefined. AUC is undefined if only one observed outcome class exists.

### Fairness lens definitions

For each user-defined group:

- **Demographic / positive parity gap:** `max_g P(predicted=1|group=g) - min_g P(predicted=1|group=g)`.
- **Equal opportunity proxy / recall gap:** `max_g TPR_g - min_g TPR_g` among groups with defined TPR.
- **False positive rate gap:** `max_g FPR_g - min_g FPR_g` among groups with defined FPR.

These are **absolute range differences**, not statistical significance tests, causal effects or legal standards. The UI warns when any cohort contains fewer than 30 observations or has only one outcome class. Such warnings are practical heuristics, not a universal threshold for adequate inference. Comparisons may be unreliable even with larger samples.

### Threshold analysis

The dashboard evaluates thresholds `0.00, 0.05, ... 1.00` for its comparison table. It identifies the threshold with the largest **observed in-sample F1** among these 21 points (ties broken in favor of the smaller threshold). Because this is determined on the same evaluation data, it is exploratory and **not a validated, cost-aware, safety-aware or fairness-aware deployment recommendation**. The slider itself permits increments of 0.01.

### Bootstrap

The *Calculate 95% interval* control performs 240 deterministic row-level bootstrap resamples with replacement, using PRNG seed `24`. It computes the empirical F1 distribution at the current threshold and displays percentile bounds at 2.5% and 97.5%, provided enough replicates produce a defined value. Intervals are conditional on independent and identically distributed row sampling; they may be misleading for time series, clusters, distribution shift, leakage or biased observational samples.

## Synthetic examples

Three reproducible 720-record examples are programmatically generated with a seeded PRNG. Samples use the generic cohort labels **North**, **Central**, **South**. The data include scores, synthetic outcomes and minor cohort-linked shifts to illustrate auditing. They do not represent any measured model, protected characteristic, real demographic data or customer group.

## Project structure

```text
README.md       Hugging Face Space metadata and full documentation
index.html      Accessible dashboard layout and all four modules
styles.css      Responsive dashboard design without external assets
core.mjs        Auditable evaluation mathematics, CSV validation and simulation
app.mjs         Browser interaction, charts, privacy-safe exports
LICENSE         MIT license
```

Source bundle additionally includes `tests/core.test.mjs`, `tests/browser_check.py` and browser screenshots. The deployable ZIP contains **only the six root files** listed above.

## Run locally and test

You can open a local static web server (ES modules require a normal HTTP origin):

```bash
python -m http.server 8000
# Open http://localhost:8000
```

Run JavaScript unit tests with Node.js 20+:

```bash
node --test tests/core.test.mjs
```

The optional Playwright smoke test uses Python Playwright and Chromium. It embeds the same CSS and JS into an isolated browser document to avoid requiring Internet access.

## Reproducibility and provenance

- Evaluation logic: `core.mjs`.
- Demonstration sample generation: `syntheticRows()` with fixed seed `2026`.
- Bootstrap confidence interval: `bootstrapInterval()` with fixed seed `24`.
- CSV parser, diagnostics, charts and downloadable reports: no third-party computation service.
- Licensed under MIT. See `LICENSE`.

## Limitations

PrismAudit is a professional **evaluation and educational prototype**, not an independently validated risk management system. No model cards, dataset cards, signed model artifacts, CI/CD deployment or out-of-sample holdout guarantee are provided by this Space. AUC, Brier score and calibration metrics assume that scores and labels refer to the same positive event; selecting a different label polarity changes their meaning. A visually small gap does not prove fairness, and a visually large gap does not establish its cause. Independent testing, domain-specific impact analysis, privacy review and expert validation remain necessary before high-stakes use.

---

**Project 24 · [Hugdrigo](https://huggingface.co/Hugdrigo) · MIT License**