README.md

metadata

title: PrismAudit | Responsible ML Evaluation Studio
emoji: 🔎
colorFrom: green
colorTo: blue
sdk: static
app_file: index.html
pinned: false
license: mit
short_description: Local-first classifier evaluation and fairness analysis
tags:
  - machine-learning
  - model-evaluation
  - classification
  - responsible-ai
  - explainability
  - fairness
  - data-analysis
  - javascript

PrismAudit | Responsible ML Evaluation Studio

Project 24 of the Hugdrigo portfolio · An open-source, local-first studio for evaluating binary classification scores, understanding decision thresholds, checking probability calibration and measuring differences across user-defined cohorts.

Open the Space

Interpretation limits: PrismAudit does not train a machine-learning model, predict new scores, prove fairness or detect unlawful discrimination. It evaluates existing binary scores against known labels. Built-in sample data are fictional and synthetic, not measurements of any real company, people or deployed model. Reports are descriptive and do not constitute validation for deployment or legal compliance.

Overview

Many classification dashboards stop at accuracy, hiding important trade-offs and subgroup differences. PrismAudit turns an existing set of probability scores into an interpretable, reproducible evaluation report that runs entirely in a browser.

Module	Functions
Overview	Accuracy, F1, ROC AUC, Brier score; confusion matrix, ROC plot, 10-bin calibration diagnostic and explainable audit signals.
Threshold explorer	Interactive decision cutoff (0.00–1.00); precision, recall, specificity and balanced accuracy; 21-point sensitivity sweep; highest in-sample F1 operating point; seeded bootstrap interval.
Fairness lens	Cohort decision rates, prevalence, recall, false positive rates and absolute max–min differences, with sample-size and missing-class warnings.
Data & methods	Local CSV import and schema validation, dataset profiling, synthetic example download and transparent methodology.
Export	Reproducible JSON audit with all aggregate metrics; CSV operating-point and cohort tables.

No paid infrastructure

• Runs as an ordinary Hugging Face Static Space (sdk: static).
• Uses vanilla HTML, CSS and ECMAScript modules. No framework, API key, API calls, build pipeline, npm packages or cloud processing.
• CSV processing, metric computation, chart rendering and export happen in browser memory.
• No analytics, CDN, trackers, external telemetry or network requests are initiated by the application itself. Hosting-provider logging may still occur when accessing the Space.

Start here

1. Open the App tab in Hugging Face.
2. Explore a fictional sample: Support prioritization, Quality review or Training completion.
3. Move the threshold slider and observe the changing confusion matrix, F1, precision, recall and group metrics. AUC, calibration and Brier score remain threshold-independent.
4. Use Threshold explorer to view all operating points and a seeded bootstrap interval.
5. Use Fairness lens for disaggregated cohort rates. Treat differences as prompts for investigation, not findings of unfairness.
6. Under Data & methods, import your own CSV, or download the example CSV to see the expected format.
7. Export a JSON audit and selected CSV tables with dedicated buttons.

CSV contract

Required columns (case-insensitive header names):

Column	Example	Definition
score	0.86	Numeric model-assigned score in the closed interval [0, 1]. Interpreted as an estimated positive-class probability for calibration analysis.
label	1	Known ground-truth binary outcome: 0 or 1.
group	North	Categorical cohort assignment, nonempty, maximum 72 characters.
id	CASE-001	Optional row identifier; not required for any metric.

Example:

id,score,label,group
CASE-001,0.86,1,North
CASE-002,0.22,0,South
CASE-003,0.62,0,North
CASE-004,0.14,1,Central

Requirements and behavior:

• File extension .csv, maximum 2 MiB; at most 10,000 data rows and 35 distinct cohorts.
• Supports UTF-8 CSV files with , delimiters, CRLF line endings, quoted fields, doubled quotes and embedded line breaks in quoted fields.
• Rejects missing or duplicate required headers, missing groups, non-binary labels, invalid scores and malformed row lengths with actionable messages.
• No scores are produced, adjusted or retrained. The slider changes only the positive/negative decision cutoff.
• CSV rows are not included in JSON audit exports. Aggregate metrics and the imported file name may be present in the report, so review outputs before sharing.
• Samples with very few rows or only one label class can yield undefined metrics. The UI uses — and JSON uses null, not zero, in these cases.

Dataset privacy

Imported data are parsed in the browser. PrismAudit's own code does not upload the file to any server. It does not persist data in cookies or browser storage. The application is not a substitute for an organization's data governance controls: prefer sanitized or synthetic datasets when evaluating potentially personal, regulated or confidential data.

Methods and equations

Let TP, FP, TN, FN be the confusion counts at a chosen threshold t, with predicted positive iff score >= t.

Measure	Definition
Accuracy	(TP + TN) / N
Precision	TP / (TP + FP)
Recall / true positive rate	TP / (TP + FN)
Specificity / true negative rate	TN / (TN + FP)
False positive rate	FP / (FP + TN)
F1	2 * precision * recall / (precision + recall)
Balanced accuracy	(recall + specificity) / 2
Positive prediction rate	(TP + FP) / N
Brier score	mean((score - label)^2)
ECE	sum_b((n_b / N) * abs(mean_score_b - observed_positive_rate_b)) for 10 equal-width score bins
ROC AUC	Tie-aware Mann–Whitney rank statistic, equivalent to ROC AUC for these empirical binary-score samples

Division with a zero denominator yields an undefined rate. An F1 of zero is returned for defined zero-valued precision or recall; otherwise undefined inputs remain undefined. AUC is undefined if only one observed outcome class exists.

Fairness lens definitions

For each user-defined group:

• Demographic / positive parity gap: max_g P(predicted=1|group=g) - min_g P(predicted=1|group=g).
• Equal opportunity proxy / recall gap: max_g TPR_g - min_g TPR_g among groups with defined TPR.
• False positive rate gap: max_g FPR_g - min_g FPR_g among groups with defined FPR.

These are absolute range differences, not statistical significance tests, causal effects or legal standards. The UI warns when any cohort contains fewer than 30 observations or has only one outcome class. Such warnings are practical heuristics, not a universal threshold for adequate inference. Comparisons may be unreliable even with larger samples.

Threshold analysis

The dashboard evaluates thresholds 0.00, 0.05, ... 1.00 for its comparison table. It identifies the threshold with the largest observed in-sample F1 among these 21 points (ties broken in favor of the smaller threshold). Because this is determined on the same evaluation data, it is exploratory and not a validated, cost-aware, safety-aware or fairness-aware deployment recommendation. The slider itself permits increments of 0.01.

Bootstrap

The Calculate 95% interval control performs 240 deterministic row-level bootstrap resamples with replacement, using PRNG seed 24. It computes the empirical F1 distribution at the current threshold and displays percentile bounds at 2.5% and 97.5%, provided enough replicates produce a defined value. Intervals are conditional on independent and identically distributed row sampling; they may be misleading for time series, clusters, distribution shift, leakage or biased observational samples.

Synthetic examples

Three reproducible 720-record examples are programmatically generated with a seeded PRNG. Samples use the generic cohort labels North, Central, South. The data include scores, synthetic outcomes and minor cohort-linked shifts to illustrate auditing. They do not represent any measured model, protected characteristic, real demographic data or customer group.

Project structure

README.md       Hugging Face Space metadata and full documentation
index.html      Accessible dashboard layout and all four modules
styles.css      Responsive dashboard design without external assets
core.mjs        Auditable evaluation mathematics, CSV validation and simulation
app.mjs         Browser interaction, charts, privacy-safe exports
LICENSE         MIT license

Source bundle additionally includes tests/core.test.mjs, tests/browser_check.py and browser screenshots. The deployable ZIP contains only the six root files listed above.

Run locally and test

You can open a local static web server (ES modules require a normal HTTP origin):

python -m http.server 8000
# Open http://localhost:8000

Run JavaScript unit tests with Node.js 20+:

node --test tests/core.test.mjs

The optional Playwright smoke test uses Python Playwright and Chromium. It embeds the same CSS and JS into an isolated browser document to avoid requiring Internet access.

Reproducibility and provenance

• Evaluation logic: core.mjs.
• Demonstration sample generation: syntheticRows() with fixed seed 2026.
• Bootstrap confidence interval: bootstrapInterval() with fixed seed 24.
• CSV parser, diagnostics, charts and downloadable reports: no third-party computation service.
• Licensed under MIT. See LICENSE.

Limitations

PrismAudit is a professional evaluation and educational prototype, not an independently validated risk management system. No model cards, dataset cards, signed model artifacts, CI/CD deployment or out-of-sample holdout guarantee are provided by this Space. AUC, Brier score and calibration metrics assume that scores and labels refer to the same positive event; selecting a different label polarity changes their meaning. A visually small gap does not prove fairness, and a visually large gap does not establish its cause. Independent testing, domain-specific impact analysis, privacy review and expert validation remain necessary before high-stakes use.

---

Project 24 · Hugdrigo · MIT License