Spaces:
Running
Running
| title: ModelGuard Observatory | |
| emoji: 🔬 | |
| colorFrom: green | |
| colorTo: blue | |
| sdk: static | |
| app_file: index.html | |
| license: mit | |
| short_description: Reproducible NLP drift, errors and release-gate audits. | |
| tags: | |
| - nlp | |
| - machine-learning | |
| - model-evaluation | |
| - data-drift | |
| - explainability | |
| - multilingual | |
| - research | |
| pinned: false | |
| # ModelGuard Observatory | Reproducible ML Evaluation Lab | |
| **Research project by Rodrigo Pantoja** · Free browser-only Hugging Face Static Space. | |
| ModelGuard Observatory explores the reproducibility, robustness, and documentation of **four deterministic text classifiers** on **184 distinct synthetic-curated evaluation examples** (130 controlled IID plus 54 challenge). Predictions are recorded for every model/example pair, yielding 736 comparison records. This is an **evaluation snapshot observatory**, not a deployed-model live monitoring service. | |
| ## Eight interactive modules | |
| 1. **Overview:** accuracy, macro-F1, generalization gap, model comparisons. | |
| 2. **Drift detection:** binned text-length PSI, label/language Jensen–Shannon divergence. | |
| 3. **Slice analysis:** descriptive subgroup outcomes by language, domain, true class. | |
| 4. **Error explorer:** confusion matrix and original evaluation examples. | |
| 5. **Model regression:** paired model comparisons and seeded bootstrap intervals. | |
| 6. **Release gates:** transparent, adjustable exploratory thresholds with failure explanations. | |
| 7. **Run registry:** model provenance, input hashes, dataset integrity and evaluation ledger. | |
| 8. **Export & governance:** reproducible CSV, JSON, Markdown and local snapshot import. | |
| ## Results and honest scope | |
| | Baseline | Controlled accuracy (130) | Novel-pattern challenge accuracy (54) | | |
| | --- | ---: | ---: | | |
| | Multinomial Naive Bayes | 100.0% | 74.1% | | |
| | Word TF-IDF | 98.5% | 72.2% | | |
| | Character TF-IDF | 95.4% | 70.4% | | |
| | Hybrid TF-IDF | 97.7% | 77.8% | | |
| These numbers were recomputed from the original **Academic Rhetorical Intent Mini v1.1** synthetic-curated dataset using the same deterministic four-model engine as **ModelOps Research Lab v2.0**. The challenge split purposely contains novel wording. Results are descriptive, **not** scientific evidence of external deployment accuracy, fairness, or safety. | |
| The initial release checklist intentionally fails its text-length PSI gate for all four methods. The tool never hides failed checks or silently overrides thresholds. | |
| ## Use | |
| Open `App`; switch between eight modules. Select a model and evaluation split. Under **Export & governance**, download original predictions, import another compatible synthetic evaluation file, or restore the bundled evidence. Imported data is processed **in your browser only**. Avoid importing personal or confidential records. | |
| The application does not automatically retrain, monitor a production service, submit external jobs, or access credentials. | |
| ## Reproduction | |
| ```bash | |
| node build_artifacts.js | |
| node test_engine.js | |
| ``` | |
| This package includes the original training and evaluation splits, their source engine, the new monitoring engine, and the source manifest with hashes. Browser workflows can be checked with `test_browser.py` (Playwright plus local Chromium), though this optional script may require setting the local Chromium executable path. | |
| ## Hosting | |
| Static Space, no servers or paid APIs. The README metadata includes `sdk: static` and `app_file: index.html`. See `DEPLOYMENT_ES.md` for a simple Mac upload guide. | |
| ## Documentation | |
| - [`METHODS.md`](METHODS.md) for definitions, formulas, and warning thresholds. | |
| - [`DATA_PROVENANCE.md`](DATA_PROVENANCE.md) and [`MODEL_CARD.md`](MODEL_CARD.md) for source details and limitations. | |
| - [`GOVERNANCE.md`](GOVERNANCE.md), [`LIMITATIONS.md`](LIMITATIONS.md), and [`SECURITY.md`](SECURITY.md) for responsible use. | |
| - [`QA_REPORT.md`](QA_REPORT.md) for test coverage and deployment status. | |
| - [`REPRODUCE.md`](REPRODUCE.md) for commands and artifact lineage. | |
| ## Licenses | |
| Application code: MIT. Original synthetic-curated dataset: CC BY 4.0, separate from the MIT code license. See `DATA_LICENSE.md`. Example text is not retrieved from journal articles or student work. | |