Spaces:
Runtime error
Runtime error
Commit History
-
feat(dictation): word-level dictation scoring with tolerant normalization (#8)
Arthur_DiazUnverified signature## What & why The ML core of M2 dictation: compares the learner's typed answer to the reference and produces a word-level diagnosis. Pure and network-free, reused by M5 (pronunciation). ## Design - Tolerance calibrated to what dictation tests — hearing/spelling the right word. Case, punctuation, spacing ignored; contractions ("don't" = "do not") and US/UK spelling ("colour" = "color") accepted as transcription variants; the word itself counts ("sheep" vs "ship") — the signal dictation must catch. - Word-aligned feedback via Levenshtein backtrace (jiwer): reference shown with substitutions/omissions/insertions marked, plus WER and an error breakdown. ## Changes - services/dictation.py: normalize_words, score_dictation → DictationResult (WER, S/D/I/H counts, per-word ops) - main.py: render_dictation_feedback (marked reference + breakdown), pure+tested - Tests: +8 (82 total) — normalization, perfect-despite-variants, phonetic confusion caught, deletion/insertion alignment, empty hypot…f6bff07 -
feat(asr): faster-whisper ASR client behind the ASRClient Protocol (#7)
Arthur_DiazUnverified signature## What & why Implements the M2 ASR service: faster-whisper (CTranslate2, int8, no torch) behind the existing ASRClient Protocol, with a config-driven factory mirroring create_llm_client. Model = small, per the PR-1 latency benchmark (ADR 0004). ## Design - Lazy model load (construction is free; the ~480 MB model loads on first use) - Synchronous transcription runs in a worker thread (asyncio.to_thread) so it never blocks the event loop - All failures (model load, decode, backend) are wrapped in ASRError — the app surfaces a clean message instead of crashing - Decoding via faster-whisper's bundled ffmpeg (file path in, no soundfile in the request path) ## Changes - services/asr/faster_whisper_client.py: FasterWhisperASRClient - services/asr/factory.py: create_asr_client (fake | faster_whisper) - services/asr/base.py: +ASRError - config/.env: ASR_PROVIDER, ASR_MODEL, ASR_COMPUTE_TYPE, ASR_CPU_THREADS - Diagnostics: "ASR quick check" (upload/mic → transcription; lazy, graceful) -…e667bd0 -
feat(asr): faster-whisper CPU latency benchmark and M2 listening ADR (#6)
Arthur_DiazUnverified signature## What & why Opens M2 (Listening). De-risks the ASR model choice before any UI is built: benchmarks faster-whisper tiny/base/small on CPU at 2 threads (≈ Space cpu-basic) for latency (p50/p95, RTF) and WER, and records the M2 architecture decisions (ADR 0004: browser TTS, faster-whisper ASR behind a Protocol, two exercise types, empirical model sizing). ## Result All three sizes clear the p95 < 10s bar → "largest under the bar" selects **small** (p95 5.18s, RTF 0.47, WER 0.9% on LibriSpeech test-clean). Accuracy headroom is a deliberate safety margin against accented non-native speech (harder than the benchmark); latency allows small comfortably. Fallback to base documented if Space cold-start (~480 MB) is too slow. ## Changes - training/benchmark_asr.py: latency+WER benchmark; decodes raw FLAC via faster-whisper's bundled ffmpeg (no torch/torchcodec, per ADR 0004) - docs/evals/m2_asr_latency.md: numbers + decision - docs/adr/0004-listening-tts-asr.md: M2 brick decisions - deps: `…37931c6 -
feat(reading): add cloze (fill-in-the-blank) as a second exercise type (#5)
Arthur_DiazUnverified signature## What & why Adds fill-in-the-blank (cloze) exercises alongside comprehension questions, selectable per generation. The verified text is shared between activity types (generated once, cached), so a cloze on an already-seen text costs only the cloze call. Blanks are chosen pedagogically by the LLM — verbs, prepositions, connectors, collocations, the levers of real mastery — with plausible distractors of the same kind. ## Robustness The LLM is not trusted for indexing. `locate_blanks` validates every proposed answer as a verbatim whole-word match against the text, computes the character offset Python-side, consumes repeated words at distinct positions, and drops anything not found — the cloze analogue of the ONNX fp32 faithfulness gate. One malformed-JSON retry, then a clear error. Free-text scoring is case/accent tolerant ("Café " matches "cafe"). ## UX - Cloze hides the intact text (it would reveal every answer) and shows only the gapped version; comprehension-questions mode still s…8a8cb44
-
feat(ml): ONNX int8 export and torch-free CPU inference service (#4)
Arthur_DiazUnverified signature## What & why Ships the M1 production model. The trained checkpoint (`en_chunked_weighted`, see docs/evals/m1_classifier_report.md) is exported to ONNX with dynamic int8 quantization and served by a **torch-free runtime service** — onnxruntime + tokenizers only, so the Space image never carries torch/transformers (~100 MB of deps instead of ~2 GB). The service reuses the exact training-time chunker and expected-rank aggregation from `src/tutor/ml/cefr/`: zero train/serve skew by construction. The artifact is distributed through a HF model repo and downloaded at startup (`CEFR_MODEL_ID`), keeping weights out of git and out of the Docker image. ## Changes - `src/tutor/ml/cefr/inference.py`: `CEFRClassifier` (`from_dir` / `from_hub`), tokenizer.json-based tokenization, batched ONNX forward, softmax + feed helpers, settings-driven factory - `training/export_onnx.py`: export with a **fp32 faithfulness gate** (the script fails if max |fp32 − torch| logit diff > 1e-2), dynamo exporter wi…933025e -
docs(evals): M1 classifier report, per-corpus OOD reads and production decision (#3)
Arthur_DiazUnverified signature## What & why Consolidates the M1 training campaign (3 arms + 2 generalization audits + published baseline, all on identical EN test documents) into the evaluation report, and records the production decision: en_chunked_weighted. ## Changes - docs/evals/m1_classifier_report.md: comparison tables, ablation reading, multilingual verdict, per-corpus domain-shift quantification, advantaged- baseline analysis, limitations - training/configs/eval_{cambridge,elg}_only.toml: evaluation-only configs for pure per-corpus OOD reads (same test docs via per-stratum seeding) - docs/adr/0003: production decision recorded ## Headline Doc-level (n=99): macro-F1 0.798 / QWK 0.936 / adjacent 0.990 — chunk-aggregate beats truncation by +0.14 accuracy. Domain shift quantified per corpus: −16–19 pts exact accuracy on an unseen source, adjacent accuracy untouched (graceful degradation). The published baseline (likely advantaged by train/test overlap) reaches 0.403 macro-F1 through our pipeline and 0.579…683fb05 -
feat(ml): CEFR dataset builder and XLM-R training pipeline with MLflow tracking (#2)
Arthur_DiazUnverified signature## What & why Implements the ADR 0003 policies as tested, dependency-free code plus a config-driven training pipeline for the M1 CEFR classifier (experiment arms 1–3 and the generalization audits). Preprocessing, splitting, aggregation and metrics live in `src/tutor/ml/cefr/` so Space inference will apply them byte-for-byte (no train/serve skew) and CI tests them without torch; GPU-only code stays in `training/` and never enters the runtime image (ADR 0001). ## Changes - `src/tutor/ml/cefr/`: label policy (X+ → X, drop list), sentence-packing chunker, document-level stratified deterministic split, expected-rank chunk→document aggregation, pure ordinal metrics (macro-F1, adjacent accuracy, QWK, per-level F1 + support) — no sklearn - `training/`: UniversalCEFR loading, `train_cefr.py` (TOML configs, class weights, MLflow tracking, doc+sentence test views with EN headline scope, `--dry-run` with built-in leakage check), `eval_cefr.py` (evaluates any checkpoint incl. the published…14e67ea
-
feat(data): UniversalCEFR EDA report and M1 training-mix decision (#1)
Arthur_DiazUnverified signature## What & why Discovers and profiles the UniversalCEFR subsets to fix the M1 training mix (reading-level classifier). The committed EDA reports are the evidence behind the updated ADR 0003. ## Changes - scripts/eda_universalcefr.py: org discovery, per-subset stats, markdown report (replaces the M0 placeholder download_data.py) - docs/evals/m1_data_eda.md + m1_data_eda_all.md: committed reports (EN + all) - docs/adr/0003: training-mix decision + multilingual addendum - pyproject/uv.lock: huggingface-hub added to the data group ## Decision - In (reference only): cambridge_exams_en, elg_cefr_en, cefr_sp_en, readme_en; non-EN reference subsets as a *measured* experiment arm (+13.6k rows) - Out: icle500_en, cefr_asag_en (learner production → M3) - Labels: X+ → X (recovers 114 en + 1,505 nl rows); drop NA / EMPTY / unrated / Unassessable / bare A-B-C - Unit: passage (docs chunked ~200 words, label inherited); split at document level, stratified corpus×level, fixed seed - Key constr…aaf2b86 -
feat: M0 walking skeleton (app, service protocols, CI/CD, Space deploy)
- Gradio app: one tab per skill + diagnostics tab with LLM ping - Provider-agnostic services via typing.Protocol + tested fakes (single OpenAI-compatible LLM client: Gemini/Mistral/Ollama/OpenAI) - Env-driven config (pydantic-settings), offline-safe defaults - CI: ruff + pytest, Docker image pushed to GHCR on main - Repo sync workflow to HF Space (Docker SDK, cpu-basic) - ADRs: stack & architecture, LLM provider strategy, datasets & licensing00d5ae5 -
Initial commit
Arthur_DiazUnverified signature6909b9e