Spaces:
Runtime error
Runtime error
Commit History
-
feat(asr): faster-whisper CPU latency benchmark and M2 listening ADR (#6)
Arthur_DiazUnverified signature## What & why Opens M2 (Listening). De-risks the ASR model choice before any UI is built: benchmarks faster-whisper tiny/base/small on CPU at 2 threads (≈ Space cpu-basic) for latency (p50/p95, RTF) and WER, and records the M2 architecture decisions (ADR 0004: browser TTS, faster-whisper ASR behind a Protocol, two exercise types, empirical model sizing). ## Result All three sizes clear the p95 < 10s bar → "largest under the bar" selects **small** (p95 5.18s, RTF 0.47, WER 0.9% on LibriSpeech test-clean). Accuracy headroom is a deliberate safety margin against accented non-native speech (harder than the benchmark); latency allows small comfortably. Fallback to base documented if Space cold-start (~480 MB) is too slow. ## Changes - training/benchmark_asr.py: latency+WER benchmark; decodes raw FLAC via faster-whisper's bundled ffmpeg (no torch/torchcodec, per ADR 0004) - docs/evals/m2_asr_latency.md: numbers + decision - docs/adr/0004-listening-tts-asr.md: M2 brick decisions - deps: `…37931c6
-
feat(ml): ONNX int8 export and torch-free CPU inference service (#4)
Arthur_DiazUnverified signature## What & why Ships the M1 production model. The trained checkpoint (`en_chunked_weighted`, see docs/evals/m1_classifier_report.md) is exported to ONNX with dynamic int8 quantization and served by a **torch-free runtime service** — onnxruntime + tokenizers only, so the Space image never carries torch/transformers (~100 MB of deps instead of ~2 GB). The service reuses the exact training-time chunker and expected-rank aggregation from `src/tutor/ml/cefr/`: zero train/serve skew by construction. The artifact is distributed through a HF model repo and downloaded at startup (`CEFR_MODEL_ID`), keeping weights out of git and out of the Docker image. ## Changes - `src/tutor/ml/cefr/inference.py`: `CEFRClassifier` (`from_dir` / `from_hub`), tokenizer.json-based tokenization, batched ONNX forward, softmax + feed helpers, settings-driven factory - `training/export_onnx.py`: export with a **fp32 faithfulness gate** (the script fails if max |fp32 − torch| logit diff > 1e-2), dynamo exporter wi…933025e -
docs(evals): M1 classifier report, per-corpus OOD reads and production decision (#3)
Arthur_DiazUnverified signature## What & why Consolidates the M1 training campaign (3 arms + 2 generalization audits + published baseline, all on identical EN test documents) into the evaluation report, and records the production decision: en_chunked_weighted. ## Changes - docs/evals/m1_classifier_report.md: comparison tables, ablation reading, multilingual verdict, per-corpus domain-shift quantification, advantaged- baseline analysis, limitations - training/configs/eval_{cambridge,elg}_only.toml: evaluation-only configs for pure per-corpus OOD reads (same test docs via per-stratum seeding) - docs/adr/0003: production decision recorded ## Headline Doc-level (n=99): macro-F1 0.798 / QWK 0.936 / adjacent 0.990 — chunk-aggregate beats truncation by +0.14 accuracy. Domain shift quantified per corpus: −16–19 pts exact accuracy on an unseen source, adjacent accuracy untouched (graceful degradation). The published baseline (likely advantaged by train/test overlap) reaches 0.403 macro-F1 through our pipeline and 0.579…683fb05 -
feat(ml): CEFR dataset builder and XLM-R training pipeline with MLflow tracking (#2)
Arthur_DiazUnverified signature## What & why Implements the ADR 0003 policies as tested, dependency-free code plus a config-driven training pipeline for the M1 CEFR classifier (experiment arms 1–3 and the generalization audits). Preprocessing, splitting, aggregation and metrics live in `src/tutor/ml/cefr/` so Space inference will apply them byte-for-byte (no train/serve skew) and CI tests them without torch; GPU-only code stays in `training/` and never enters the runtime image (ADR 0001). ## Changes - `src/tutor/ml/cefr/`: label policy (X+ → X, drop list), sentence-packing chunker, document-level stratified deterministic split, expected-rank chunk→document aggregation, pure ordinal metrics (macro-F1, adjacent accuracy, QWK, per-level F1 + support) — no sklearn - `training/`: UniversalCEFR loading, `train_cefr.py` (TOML configs, class weights, MLflow tracking, doc+sentence test views with EN headline scope, `--dry-run` with built-in leakage check), `eval_cefr.py` (evaluates any checkpoint incl. the published…14e67ea
-
feat: M0 walking skeleton (app, service protocols, CI/CD, Space deploy)
- Gradio app: one tab per skill + diagnostics tab with LLM ping - Provider-agnostic services via typing.Protocol + tested fakes (single OpenAI-compatible LLM client: Gemini/Mistral/Ollama/OpenAI) - Env-driven config (pydantic-settings), offline-safe defaults - CI: ruff + pytest, Docker image pushed to GHCR on main - Repo sync workflow to HF Space (Docker SDK, cpu-basic) - ADRs: stack & architecture, LLM provider strategy, datasets & licensing00d5ae5