README.md

metadata

title: ModelOps Research Lab
emoji: 🧪
colorFrom: indigo
colorTo: blue
sdk: static
app_file: index.html
pinned: false
license: mit
short_description: Reproducible multilingual NLP model evaluation and audits.
tags:
  - static
  - model-evaluation
  - nlp
  - multilingual
  - reproducibility
  - experiment-tracking
  - explainable-ai

🧪 ModelOps Research Lab v2.0

An offline-first, zero-cost browser application for reproducible multilingual text-classification experiments, created for the applied AI research portfolio of Rodrigo Pantoja (@Hugdrigo).

Purpose

Move beyond attractive demos to measured model behavior: training, fixed evaluation partitions, experimental comparisons, error inspection, language/domain slices, bootstrap intervals, source audits, and traceable result exports.

Quickstart

1. Open App. The four transparent lexical baselines are fitted in your browser on the included 842 synthetic training examples.
2. See Overview for the 54-item novel-pattern challenge, and switch to the controlled 130-item IID test.
3. Open Experiments and click Run four baselines to create a locally saved run with seed-controlled bootstrap sampling.
4. Visit Slice analysis, Error lab, Shift diagnostics and Data audit.
5. Export predictions (CSV), metrics (JSON), reports (Markdown), and run history (JSON).

Exact research data and licensing

Source: Academic Rhetorical Intent Mini v1.1 (originally developed as an earlier portfolio research dataset). All 1,026 examples are synthetic-curated, written for educational experimental use; no real scholarly manuscripts or student submissions were copied. Distribution: 842 training, 130 controlled IID, 54 held-out novel-pattern challenge. Languages: English, Spanish, Portuguese. Six rhetorical labels: background, objective, method, result, limitation, implication.

Dataset: CC BY 4.0. See DATA_LICENSE.md and ORIGINAL_DATASET_CARD.md. App and baseline engine: MIT. See LICENSE. The README's license: mit denotes the Space's application code and does not override the included dataset attribution.

Tested baselines and evaluation

All four algorithms are implemented in engine.js and trained from train.json on every page load:

Method	Controlled IID Accuracy (130)	Novel-pattern Accuracy (54)	Novel-pattern Macro F1
Multinomial Naive Bayes	100.0%	74.1%	73.6%
Word TF-IDF centroid	98.5%	72.2%	70.6%
Character trigram TF-IDF centroid	95.4%	70.4%	68.4%
Weighted word/character hybrid	97.7%	77.8%	76.2%

These are new browser baselines; they do not reproduce the previously trained Python model. The hybrid is a weighted combination of two baselines, not a separately trained neural network. Scores are derived from the fixed synthetic dataset, not a scientific claim of transferable academic performance. See MODEL_CARD.md, METHODS.md and LIMITATIONS.md.

A historical different Python model reported 79.6% challenge Accuracy and only 35.7% on a very small 28-sentence PubMed development smoke test. The latter is shown separately in the dashboard to remind users about domain shift, not to imply that the browser classifiers were evaluated on PubMed.

Reproduce without a browser

node --test test_engine.js
node benchmark_cli.js

Node 18+ required. No npm install needed. In-browser operation needs JavaScript and no paid API. These tests do not require internet.

Methodological notes

• No training or hyperparameter fitting uses the holdout splits.
• Method weights and feature definitions are specified in code, not optimized on the challenge.
• Macro F1 averages labels present in the evaluated group. For 130/54 full sets, all six labels are present.
• Bootstrap intervals are resampling intervals across fixed held-out rows. They are not estimates of uncertainty due to training sets, label noise or real-world deployment.
• The Data audit checks exact text / ID overlap. Semantic or template-level leakage is not excluded.
• Scores from synthetic examples must never be used for grading, employment, admissions or certification of educational quality.

Repository structure

• index.html, styles.css, app.js, engine.js: standalone Static Space app and model code.
• train.json, iid_test.json, challenge.json: original fixed splits used by the live app.
• academic_rhetoric_intent.csv: original combined synthetic dataset, with source metadata.
• dataset_manifest.json, DATA_CARD.md, MODEL_CARD.md: provenance and technical cards.
• evaluation_results.json: independently recomputed baseline metrics.
• test_engine.js, benchmark_cli.js, test_browser.py: reproducibility and browser QA.
• DEPLOYMENT_ES.md: easy Spanish uploading guide.
• Other MD files: limitations, methods, source license and release checks.

Privacy, cost and limitations

The live app sends no user sentence to any server. Model training and scoring operate locally in the browser. Short experiment-summary history uses browser local storage; exports are local downloads. The bundled texts are synthetic and public. Zero paid model inference or managed compute is required.

See SECURITY.md and LIMITATIONS.md for caveats. Please treat the application as a research demonstration, not as a grading or decision-making system.