Spaces:
Running
Running
| title: ModelOps Research Lab | |
| emoji: "🧪" | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: static | |
| app_file: index.html | |
| pinned: false | |
| license: mit | |
| short_description: Reproducible multilingual NLP model evaluation and audits. | |
| tags: | |
| - static | |
| - model-evaluation | |
| - nlp | |
| - multilingual | |
| - reproducibility | |
| - experiment-tracking | |
| - explainable-ai | |
| # 🧪 ModelOps Research Lab v2.0 | |
| An offline-first, zero-cost browser application for **reproducible multilingual text-classification experiments**, created for the applied AI research portfolio of Rodrigo Pantoja (`@Hugdrigo`). | |
| ## Purpose | |
| Move beyond attractive demos to **measured model behavior**: training, fixed evaluation partitions, experimental comparisons, error inspection, language/domain slices, bootstrap intervals, source audits, and traceable result exports. | |
| ## Quickstart | |
| 1. Open **App**. The four transparent lexical baselines are fitted *in your browser* on the included 842 synthetic training examples. | |
| 2. See **Overview** for the 54-item novel-pattern challenge, and switch to the controlled 130-item IID test. | |
| 3. Open **Experiments** and click **Run four baselines** to create a locally saved run with seed-controlled bootstrap sampling. | |
| 4. Visit **Slice analysis**, **Error lab**, **Shift diagnostics** and **Data audit**. | |
| 5. Export predictions (CSV), metrics (JSON), reports (Markdown), and run history (JSON). | |
| ## Exact research data and licensing | |
| Source: **Academic Rhetorical Intent Mini v1.1** (originally developed as an earlier portfolio research dataset). All **1,026 examples are synthetic-curated**, written for educational experimental use; no real scholarly manuscripts or student submissions were copied. Distribution: 842 training, 130 controlled IID, 54 held-out novel-pattern challenge. Languages: English, Spanish, Portuguese. Six rhetorical labels: background, objective, method, result, limitation, implication. | |
| **Dataset**: CC BY 4.0. See `DATA_LICENSE.md` and `ORIGINAL_DATASET_CARD.md`. **App and baseline engine**: MIT. See `LICENSE`. The README's `license: mit` denotes the Space's application code and does not override the included dataset attribution. | |
| ## Tested baselines and evaluation | |
| All four algorithms are implemented in `engine.js` and trained from `train.json` on every page load: | |
| | Method | Controlled IID Accuracy (130) | Novel-pattern Accuracy (54) | Novel-pattern Macro F1 | | |
| |---|---:|---:|---:| | |
| | Multinomial Naive Bayes | 100.0% | 74.1% | 73.6% | | |
| | Word TF-IDF centroid | 98.5% | 72.2% | 70.6% | | |
| | Character trigram TF-IDF centroid | 95.4% | 70.4% | 68.4% | | |
| | Weighted word/character hybrid | 97.7% | **77.8%** | 76.2% | | |
| These are **new browser baselines**; they do not reproduce the previously trained Python model. The hybrid is a weighted combination of two baselines, not a separately trained neural network. Scores are derived from the fixed **synthetic** dataset, not a scientific claim of transferable academic performance. See `MODEL_CARD.md`, `METHODS.md` and `LIMITATIONS.md`. | |
| A historical *different Python model* reported 79.6% challenge Accuracy and only 35.7% on a very small 28-sentence PubMed development smoke test. The latter is shown separately in the dashboard to remind users about domain shift, **not** to imply that the browser classifiers were evaluated on PubMed. | |
| ## Reproduce without a browser | |
| ```bash | |
| node --test test_engine.js | |
| node benchmark_cli.js | |
| ``` | |
| Node 18+ required. No npm install needed. In-browser operation needs JavaScript and no paid API. These tests do not require internet. | |
| ## Methodological notes | |
| - No training or hyperparameter fitting uses the holdout splits. | |
| - Method weights and feature definitions are specified in code, **not optimized on the challenge**. | |
| - Macro F1 averages labels present in the evaluated group. For 130/54 full sets, all six labels are present. | |
| - Bootstrap intervals are **resampling intervals across fixed held-out rows**. They are not estimates of uncertainty due to training sets, label noise or real-world deployment. | |
| - The Data audit checks exact text / ID overlap. **Semantic or template-level leakage is not excluded.** | |
| - Scores from synthetic examples must **never** be used for grading, employment, admissions or certification of educational quality. | |
| ## Repository structure | |
| - `index.html`, `styles.css`, `app.js`, `engine.js`: standalone Static Space app and model code. | |
| - `train.json`, `iid_test.json`, `challenge.json`: original fixed splits used by the live app. | |
| - `academic_rhetoric_intent.csv`: original combined synthetic dataset, with source metadata. | |
| - `dataset_manifest.json`, `DATA_CARD.md`, `MODEL_CARD.md`: provenance and technical cards. | |
| - `evaluation_results.json`: independently recomputed baseline metrics. | |
| - `test_engine.js`, `benchmark_cli.js`, `test_browser.py`: reproducibility and browser QA. | |
| - `DEPLOYMENT_ES.md`: easy Spanish uploading guide. | |
| - Other MD files: limitations, methods, source license and release checks. | |
| ## Privacy, cost and limitations | |
| The live app sends **no user sentence** to any server. Model training and scoring operate locally in the browser. Short experiment-summary history uses browser local storage; exports are local downloads. The bundled texts are synthetic and public. Zero paid model inference or managed compute is required. | |
| See `SECURITY.md` and `LIMITATIONS.md` for caveats. Please treat the application as a **research demonstration**, not as a grading or decision-making system. | |