---
title: RobustnessLab AI
emoji: "🛡️"
colorFrom: indigo
colorTo: green
sdk: static
app_file: index.html
license: mit
short_description: Auditable multilingual NLP robustness and stress tests.
tags:
  - static
  - nlp
  - multilingual
  - machine-learning
  - robustness
  - model-evaluation
  - interpretability
  - reproducibility
pinned: false
---

# 🛡️ RobustnessLab AI | Multilingual Stress & Reliability Studio

**A research-engineering portfolio project by Rodrigo Pantoja (`@Hugdrigo`)**. Runs locally in the user's browser with **zero paid APIs**, **zero server inference**, and auditable machine-learning evaluation artifacts.

## Why it exists

Accuracy on a convenient synthetic test split can hide sensitivity to input changes. This app extends **ModelOps Research Lab** and **ModelGuard Observatory** by testing four transparent academic-rhetoric classifiers under reproducible stress conditions, visualizing prediction flips and matched-subset accuracy changes, studying score-margin-based rejection, and making research limitations explicit.

## Research evidence

- Original source: **Academic Rhetorical Intent Mini v1.1**, a synthetic-curated dataset, **CC BY 4.0**.
- Fixed splits: **842 training**, **130 controlled IID**, **54 novel-pattern challenge**, total **1,026** distinct examples.
- Four browser-compatible baselines: Multinomial Naive Bayes, word TF-IDF centroid, character TF-IDF centroid, and a weighted hybrid.
- Six conditions: original, case/punctuation formatting, deterministic internal typo, small dictionary substitution, appended neutral research context, and destructive truncation.
- **4,416 auditable prediction records** = 184 evaluation examples × 4 classifiers × 6 conditions, rebuilt using `node generate_artifacts.js`.

### Example observed results (54 synthetic challenge examples)

| Model | Original accuracy | Typo flip rate | Context-sentence flip rate |
| --- | ---: | ---: | ---: |
| Multinomial Naive Bayes | 74.1% | 5.6% | 29.6% |
| Word TF-IDF | 72.2% | 3.7% | 20.4% |
| Character TF-IDF | 70.4% | 1.9% | 16.7% |
| Hybrid TF-IDF | 77.8% | 1.9% | 9.3% |

**Important:** these rates are from synthetic authored examples and deterministic transformations, not a representative human-validated perturbation benchmark. Heuristic dictionary/context edits may change semantics. Truncation can change the gold label and therefore gets **no transformed accuracy claim**. An attack may leave a particular row unchanged; the dashboard exposes this coverage explicitly and calculates modified-subset scores on truly modified records only.

## Eight interactive research modules

1. **Command center:** original accuracy, matched stress changes, flip rates, paired bootstrap summaries and provenance.
2. **Stress matrix:** compare six conditions across four classifiers, with changed-row counts.
3. **Paired examples:** inspect each original/perturbed string, label, method prediction and example ID.
4. **Risk & abstention:** score-margin ranking, error-versus-coverage curve, AURC and configurable coverage.
5. **Reliability:** Naive Bayes posterior calibration diagnostic using eight bins and descriptive ECE. Other models have **uncalibrated similarity margins**, not probabilities.
6. **Language slices:** outcomes by English, Spanish, Portuguese, domain, and source rhetorical label.
7. **Research gates:** adjustable, explainable thresholds, including cases where models fail. **NOT production certification.**
8. **Sandbox & export:** offline stress testing of the user's sentence, Markdown report, summary CSV, all-record CSV, complete JSON audit and SHA-256 registry.

## Reproduction

```bash
node generate_artifacts.js
node test_engine.js
python test_browser.py  # optional: Playwright + Chromium
```

Node 18+ required; no npm dependencies. Read [`REPRODUCE.md`](REPRODUCE.md) and [`METHODS.md`](METHODS.md) for complete definitions and reproduction steps.

## Usage and privacy

Open the **App** tab and navigate the eight research modules. Selecting a model/split/attack changes the displayed evidence. All bundled data is local; custom sandbox sentences are processed in the browser only. There are **no API keys** or paid compute dependencies. The application does not automatically retrain production models, certify safety, send messages or change linked accounts.

## Honest limitations

All underlying academic examples are **synthetic**, curated by template and manually authored patterns. The evaluation does not provide external population validity, fairness certification, calibrated probabilities for centroid baselines, robustness guarantees, or proof that perturbed source gold labels remain correct. Bootstrap intervals resample examples on a **fixed** trained model; they omit uncertainty from retraining, label errors, and real-world shift. Read [`LIMITATIONS.md`](LIMITATIONS.md), [`ATTACK_CATALOG.md`](ATTACK_CATALOG.md), and [`CALIBRATION.md`](CALIBRATION.md).

## Licenses

App source and the published Static Space: **MIT** (`LICENSE`). Bundled source dataset and its derivative stress records remain **CC BY 4.0** (`DATA_LICENSE.md`) with source attribution. `license: mit` above refers to the application code, not a relicense of the dataset.

## Documentation

- [`DEPLOYMENT_ES.md`](DEPLOYMENT_ES.md): beginner-friendly Hugging Face upload workflow (Mac).
- [`METHODS.md`](METHODS.md): metrics, denominators, intervals, calibration and design.
- [`ATTACK_CATALOG.md`](ATTACK_CATALOG.md): perturbations, semantics and coverage assumptions.
- [`DATA_PROVENANCE.md`](DATA_PROVENANCE.md), [`DATA_CARD.md`](DATA_CARD.md), [`MODEL_CARD.md`](MODEL_CARD.md): lineage and limitations.
- [`RESEARCH_GATES.md`](RESEARCH_GATES.md), [`SECURITY.md`](SECURITY.md), [`CALIBRATION.md`](CALIBRATION.md): appropriate research usage.
- [`QA_REPORT.md`](QA_REPORT.md): offline tests and publication status.

**Publication status:** local tests completed. Public Hugging Face hosting and any user browser behaviors must still be checked after upload.