---
title: ModelGuard Observatory
emoji: 🔬
colorFrom: green
colorTo: blue
sdk: static
app_file: index.html
license: mit
short_description: Reproducible NLP drift, errors and release-gate audits.
tags:
  - nlp
  - machine-learning
  - model-evaluation
  - data-drift
  - explainability
  - multilingual
  - research
pinned: false
---

# ModelGuard Observatory | Reproducible ML Evaluation Lab

**Research project by Rodrigo Pantoja** · Free browser-only Hugging Face Static Space.

ModelGuard Observatory explores the reproducibility, robustness, and documentation of **four deterministic text classifiers** on **184 distinct synthetic-curated evaluation examples** (130 controlled IID plus 54 challenge). Predictions are recorded for every model/example pair, yielding 736 comparison records. This is an **evaluation snapshot observatory**, not a deployed-model live monitoring service.

## Eight interactive modules

1. **Overview:** accuracy, macro-F1, generalization gap, model comparisons.
2. **Drift detection:** binned text-length PSI, label/language Jensen–Shannon divergence.
3. **Slice analysis:** descriptive subgroup outcomes by language, domain, true class.
4. **Error explorer:** confusion matrix and original evaluation examples.
5. **Model regression:** paired model comparisons and seeded bootstrap intervals.
6. **Release gates:** transparent, adjustable exploratory thresholds with failure explanations.
7. **Run registry:** model provenance, input hashes, dataset integrity and evaluation ledger.
8. **Export & governance:** reproducible CSV, JSON, Markdown and local snapshot import.

## Results and honest scope

| Baseline | Controlled accuracy (130) | Novel-pattern challenge accuracy (54) |
| --- | ---: | ---: |
| Multinomial Naive Bayes | 100.0% | 74.1% |
| Word TF-IDF | 98.5% | 72.2% |
| Character TF-IDF | 95.4% | 70.4% |
| Hybrid TF-IDF | 97.7% | 77.8% |

These numbers were recomputed from the original **Academic Rhetorical Intent Mini v1.1** synthetic-curated dataset using the same deterministic four-model engine as **ModelOps Research Lab v2.0**. The challenge split purposely contains novel wording. Results are descriptive, **not** scientific evidence of external deployment accuracy, fairness, or safety.

The initial release checklist intentionally fails its text-length PSI gate for all four methods. The tool never hides failed checks or silently overrides thresholds.

## Use

Open `App`; switch between eight modules. Select a model and evaluation split. Under **Export & governance**, download original predictions, import another compatible synthetic evaluation file, or restore the bundled evidence. Imported data is processed **in your browser only**. Avoid importing personal or confidential records.

The application does not automatically retrain, monitor a production service, submit external jobs, or access credentials.

## Reproduction

```bash
node build_artifacts.js
node test_engine.js
```

This package includes the original training and evaluation splits, their source engine, the new monitoring engine, and the source manifest with hashes. Browser workflows can be checked with `test_browser.py` (Playwright plus local Chromium), though this optional script may require setting the local Chromium executable path.

## Hosting

Static Space, no servers or paid APIs. The README metadata includes `sdk: static` and `app_file: index.html`. See `DEPLOYMENT_ES.md` for a simple Mac upload guide.

## Documentation

- [`METHODS.md`](METHODS.md) for definitions, formulas, and warning thresholds.
- [`DATA_PROVENANCE.md`](DATA_PROVENANCE.md) and [`MODEL_CARD.md`](MODEL_CARD.md) for source details and limitations.
- [`GOVERNANCE.md`](GOVERNANCE.md), [`LIMITATIONS.md`](LIMITATIONS.md), and [`SECURITY.md`](SECURITY.md) for responsible use.
- [`QA_REPORT.md`](QA_REPORT.md) for test coverage and deployment status.
- [`REPRODUCE.md`](REPRODUCE.md) for commands and artifact lineage.

## Licenses

Application code: MIT. Original synthetic-curated dataset: CC BY 4.0, separate from the MIT code license. See `DATA_LICENSE.md`. Example text is not retrieved from journal articles or student work.