README.md

metadata

title: ModelGuard Observatory
emoji: 🔬
colorFrom: green
colorTo: blue
sdk: static
app_file: index.html
license: mit
short_description: Reproducible NLP drift, errors and release-gate audits.
tags:
  - nlp
  - machine-learning
  - model-evaluation
  - data-drift
  - explainability
  - multilingual
  - research
pinned: false

ModelGuard Observatory | Reproducible ML Evaluation Lab

Research project by Rodrigo Pantoja · Free browser-only Hugging Face Static Space.

ModelGuard Observatory explores the reproducibility, robustness, and documentation of four deterministic text classifiers on 184 distinct synthetic-curated evaluation examples (130 controlled IID plus 54 challenge). Predictions are recorded for every model/example pair, yielding 736 comparison records. This is an evaluation snapshot observatory, not a deployed-model live monitoring service.

Eight interactive modules

1. Overview: accuracy, macro-F1, generalization gap, model comparisons.
2. Drift detection: binned text-length PSI, label/language Jensen–Shannon divergence.
3. Slice analysis: descriptive subgroup outcomes by language, domain, true class.
4. Error explorer: confusion matrix and original evaluation examples.
5. Model regression: paired model comparisons and seeded bootstrap intervals.
6. Release gates: transparent, adjustable exploratory thresholds with failure explanations.
7. Run registry: model provenance, input hashes, dataset integrity and evaluation ledger.
8. Export & governance: reproducible CSV, JSON, Markdown and local snapshot import.

Results and honest scope

Baseline	Controlled accuracy (130)	Novel-pattern challenge accuracy (54)
Multinomial Naive Bayes	100.0%	74.1%
Word TF-IDF	98.5%	72.2%
Character TF-IDF	95.4%	70.4%
Hybrid TF-IDF	97.7%	77.8%

These numbers were recomputed from the original Academic Rhetorical Intent Mini v1.1 synthetic-curated dataset using the same deterministic four-model engine as ModelOps Research Lab v2.0. The challenge split purposely contains novel wording. Results are descriptive, not scientific evidence of external deployment accuracy, fairness, or safety.

The initial release checklist intentionally fails its text-length PSI gate for all four methods. The tool never hides failed checks or silently overrides thresholds.

Use

Open App; switch between eight modules. Select a model and evaluation split. Under Export & governance, download original predictions, import another compatible synthetic evaluation file, or restore the bundled evidence. Imported data is processed in your browser only. Avoid importing personal or confidential records.

The application does not automatically retrain, monitor a production service, submit external jobs, or access credentials.

Reproduction

node build_artifacts.js
node test_engine.js

This package includes the original training and evaluation splits, their source engine, the new monitoring engine, and the source manifest with hashes. Browser workflows can be checked with test_browser.py (Playwright plus local Chromium), though this optional script may require setting the local Chromium executable path.

Hosting

Static Space, no servers or paid APIs. The README metadata includes sdk: static and app_file: index.html. See DEPLOYMENT_ES.md for a simple Mac upload guide.

Documentation

• METHODS.md for definitions, formulas, and warning thresholds.
• DATA_PROVENANCE.md and MODEL_CARD.md for source details and limitations.
• GOVERNANCE.md, LIMITATIONS.md, and SECURITY.md for responsible use.
• QA_REPORT.md for test coverage and deployment status.
• REPRODUCE.md for commands and artifact lineage.

Licenses

Application code: MIT. Original synthetic-curated dataset: CC BY 4.0, separate from the MIT code license. See DATA_LICENSE.md. Example text is not retrieved from journal articles or student work.