Spaces:
Runtime error
Runtime error
Commit History
-
feat(asr): faster-whisper CPU latency benchmark and M2 listening ADR (#6)
Arthur_DiazUnverified signature## What & why Opens M2 (Listening). De-risks the ASR model choice before any UI is built: benchmarks faster-whisper tiny/base/small on CPU at 2 threads (≈ Space cpu-basic) for latency (p50/p95, RTF) and WER, and records the M2 architecture decisions (ADR 0004: browser TTS, faster-whisper ASR behind a Protocol, two exercise types, empirical model sizing). ## Result All three sizes clear the p95 < 10s bar → "largest under the bar" selects **small** (p95 5.18s, RTF 0.47, WER 0.9% on LibriSpeech test-clean). Accuracy headroom is a deliberate safety margin against accented non-native speech (harder than the benchmark); latency allows small comfortably. Fallback to base documented if Space cold-start (~480 MB) is too slow. ## Changes - training/benchmark_asr.py: latency+WER benchmark; decodes raw FLAC via faster-whisper's bundled ffmpeg (no torch/torchcodec, per ADR 0004) - docs/evals/m2_asr_latency.md: numbers + decision - docs/adr/0004-listening-tts-asr.md: M2 brick decisions - deps: `…37931c6
-
feat(ml): ONNX int8 export and torch-free CPU inference service (#4)
Arthur_DiazUnverified signature## What & why Ships the M1 production model. The trained checkpoint (`en_chunked_weighted`, see docs/evals/m1_classifier_report.md) is exported to ONNX with dynamic int8 quantization and served by a **torch-free runtime service** — onnxruntime + tokenizers only, so the Space image never carries torch/transformers (~100 MB of deps instead of ~2 GB). The service reuses the exact training-time chunker and expected-rank aggregation from `src/tutor/ml/cefr/`: zero train/serve skew by construction. The artifact is distributed through a HF model repo and downloaded at startup (`CEFR_MODEL_ID`), keeping weights out of git and out of the Docker image. ## Changes - `src/tutor/ml/cefr/inference.py`: `CEFRClassifier` (`from_dir` / `from_hub`), tokenizer.json-based tokenization, batched ONNX forward, softmax + feed helpers, settings-driven factory - `training/export_onnx.py`: export with a **fp32 faithfulness gate** (the script fails if max |fp32 − torch| logit diff > 1e-2), dynamo exporter wi…933025e -
docs(evals): M1 classifier report, per-corpus OOD reads and production decision (#3)
Arthur_DiazUnverified signature## What & why Consolidates the M1 training campaign (3 arms + 2 generalization audits + published baseline, all on identical EN test documents) into the evaluation report, and records the production decision: en_chunked_weighted. ## Changes - docs/evals/m1_classifier_report.md: comparison tables, ablation reading, multilingual verdict, per-corpus domain-shift quantification, advantaged- baseline analysis, limitations - training/configs/eval_{cambridge,elg}_only.toml: evaluation-only configs for pure per-corpus OOD reads (same test docs via per-stratum seeding) - docs/adr/0003: production decision recorded ## Headline Doc-level (n=99): macro-F1 0.798 / QWK 0.936 / adjacent 0.990 — chunk-aggregate beats truncation by +0.14 accuracy. Domain shift quantified per corpus: −16–19 pts exact accuracy on an unseen source, adjacent accuracy untouched (graceful degradation). The published baseline (likely advantaged by train/test overlap) reaches 0.403 macro-F1 through our pipeline and 0.579…683fb05
-
feat(data): UniversalCEFR EDA report and M1 training-mix decision (#1)
Arthur_DiazUnverified signature## What & why Discovers and profiles the UniversalCEFR subsets to fix the M1 training mix (reading-level classifier). The committed EDA reports are the evidence behind the updated ADR 0003. ## Changes - scripts/eda_universalcefr.py: org discovery, per-subset stats, markdown report (replaces the M0 placeholder download_data.py) - docs/evals/m1_data_eda.md + m1_data_eda_all.md: committed reports (EN + all) - docs/adr/0003: training-mix decision + multilingual addendum - pyproject/uv.lock: huggingface-hub added to the data group ## Decision - In (reference only): cambridge_exams_en, elg_cefr_en, cefr_sp_en, readme_en; non-EN reference subsets as a *measured* experiment arm (+13.6k rows) - Out: icle500_en, cefr_asag_en (learner production → M3) - Labels: X+ → X (recovers 114 en + 1,505 nl rows); drop NA / EMPTY / unrated / Unassessable / bare A-B-C - Unit: passage (docs chunked ~200 words, label inherited); split at document level, stratified corpus×level, fixed seed - Key constr…aaf2b86 -
feat: M0 walking skeleton (app, service protocols, CI/CD, Space deploy)
- Gradio app: one tab per skill + diagnostics tab with LLM ping - Provider-agnostic services via typing.Protocol + tested fakes (single OpenAI-compatible LLM client: Gemini/Mistral/Ollama/OpenAI) - Env-driven config (pydantic-settings), offline-safe defaults - CI: ruff + pytest, Docker image pushed to GHCR on main - Repo sync workflow to HF Space (Docker SDK, cpu-basic) - ADRs: stack & architecture, LLM provider strategy, datasets & licensing00d5ae5