Spaces:
Sleeping
Sleeping
Commit History
-
Phrasebook eval harness + accent/filler-tolerant matching
ous-sow +1: Claude Opus 5.5- configs/eval/phrasebook_eval.json: 85 held-out inputs (typos, code-switching, filler words, open-ended questions that must reach the LLM), written to differ from the curated aliases. - scripts/eval_phrasebook.py: scores correct / wrong / false_hit / miss per language. - _normalize now folds Latin accents (ça -> ca), drops politeness filler (please, svp, s'il vous plaît) and leading articles (du, une, some). Applied to both sides; Bambara/Pular letters (ɛ ɔ ɓ ŋ) are untouched. Hit rate on the eval set: before aliases bam 43% ful 46% + aliases bam 80% ful 77% + normaliser bam 88% ful 86% wrong=0 and false_hit=0 at every step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>ea5dc26
-
Phrasebook: multi-alias + French source keys, plus misses-script stub
jefffffff9 +1: Claude Opus 4.7Two-track vocabulary enrichment that keeps the matcher and UI unchanged: 1. Multi-alias entries. Each curated row may now carry any combination of `source`, `sources` (list), `source_fr`, `sources_fr` (list). The loader flattens them into one match-candidate per alias, all pointing at the same target translation. A typed paraphrase ("morning, sir") or French equivalent ("bonjour") hits the same Bambara/Pular target as the canonical English source. Existing single-`source` rows keep working unchanged (verified: 110 rows → 110 candidates, exact match preserved). 2. top_k now dedupes by canonical target so multi-alias rows can't crowd out the RAG few-shot slots. 3. scripts/phrasebook_misses.py — stub for the data-driven curation loop. Reads data/field_turns.jsonl, groups translate-phase rows where phrasebook=null, prints the most frequent unmet inputs. Filters by --lang and --since. --draft (LLM-proposed target per miss) is a marked TODO — wire it when …5815492
-
Forbid parenthetical glosses in LLM replies
jefffffff9 +1: Claude Opus 4.7The model was appending English literal-translation annotations (e.g. "On walli e jam? (Lit: Did you sleep in peace?)") which then got read aloud by TTS as gibberish. Two-layer fix: 1. System prompt now explicitly bans "(Lit: ...)", "(meaning ...)", and any English/French parenthetical aside, and demands 100% target-language output. 2. Defensive client-side regex strips any "(Lit/Literal/Meaning/ Translation/English/French ...)" parenthetical the model emits anyway before returning the string to TTS. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>757e833
-
Stage 4: split translate/reply UI + CPU-safe TTS + reply-not-translate prompt
jefffffff9 +1: Claude Opus 4.7- 4-box layout on both Voice and Text tabs: phrasebook translation (text + audio) is automatic on submit; "Generate reply" runs the dialect-anchored LLM only when clicked. - Shared gr.State carries the canonical input (typed text or Whisper transcript) into the reply button so we never re-transcribe. - Robust device resolution: probe cuda.device_count(), and have _synthesize retry on CPU when CUDA path raises (fixes "Torch not compiled with CUDA" on CPU-only laptops). - System prompt now explicitly tells the LLM to REPLY conversationally and reframes the curated few-shot pairs as style/orthography references only, fixing the regression where the model would echo the phrasebook target verbatim instead of replying. - README: add Stage 4 entry + update entry-points table. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>9e99c2c -
Per-turn telemetry + RAG few-shot on phrasebook miss
jefffffff9Telemetry (src/engine/turn_logger.py) Thread-safe JSONL append logger writing one self-contained row per turn to data/field_turns.jsonl (overridable via FIELD_TURNS_PATH). Captures ts, tab, langs, transcript/user_text, phrasebook hit details, llm model + ms, reply, tts ms, total ms, error. Foundation for field-test review, phrasebook hit-rate measurement, and Stage-4 training-data curation. Never raises — telemetry must not break the user-facing pipeline. RAG few-shot (src/llm/phrasebook.py + src/llm/minimal_client.py) - phrasebook.top_k(text, lang, k=3): nearest-K phrasebook entries regardless of threshold, sorted by descending fuzzy score. - MinimalClient.chat() now accepts extra_examples; _build_system_prompt appends them as a second 'Additional reference phrases relevant to the current user input' section after the 30-pair anchor list. app_minimal.py - Both run_pipeline (voice) and run_text_pipeline (text) now share a single _resolve_reply() help…064d08b -
Remove stale Qwen references from minimal-baseline docstrings
jefffffff9Runtime defaults already point to CohereLabs/aya-expanse-32b; only docstrings / usage examples still named Qwen. Cosmetic cleanup only.cc82bd8
-
Ground-zero Stages 1–3: dialect anchors + phrasebook short-circuit + Aya-Expanse
jefffffff9Stage 1 — dialect-pinned LLM client (src/llm/minimal_client.py) Plain-text replacement for GemmaClient's JSON/teacher flow. System prompt pins Bambara-Mali and Pular-Fuuta-Jallon explicitly, names forbidden neighbouring languages (Wolof, Hausa, Pulaar-Senegal, Fulfulde-Nigeria, Jula-CI), and injects a 30-pair bilingual gold list as few-shot anchoring from configs/dialect_anchors/{bambara_mali,pular_guinea}.json. Stage 2 — curated phrasebook short-circuit (src/llm/phrasebook.py) 100 Bambara + 110 Pular English-keyed pairs across greetings, family, food, farming, health, shopping, travel, clarity, time, parting. Fuzzy matched (threshold 0.88) before every LLM call; on hit returns the gold translation directly — zero drift risk, zero latency. Stage 3 — default LLM swapped to CohereLabs/aya-expanse-32b 23-language multilingual base with stronger West African coverage than Qwen 2.5-7B. Overridable via LLM_MODEL_ID. Space wiring - README frontmatter app_file: app.p…d0e28fa
-
Fix jiwer crash on post-normalisation empty refs; register SLR106/105 datasets
jefffffff9 +1: Claude Sonnet 4.6compute_metrics: pre-apply lowercasing + punctuation removal before filtering empty references — a label like "?" passes r.strip() but becomes "" after RemovePunctuation, causing jiwer to raise ValueError mid-evaluation. Pass already-normalised strings directly to jiwer.cer / jiwer.wer so no second transform is applied. web_harvester: add commented-in registry entries for OpenSLR SLR106 (10k clean Guinea Pular utterances, 49 speakers, CC BY-SA 4.0) and SLR105 (Guinea radio corpus, Pular-tagged validation split). Both are uncommented once the audio is uploaded to HF — best available Guinea Pular ASR data. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>6682858
-
Add Adlam/Pular Fula integration: transliterator + 3 new datasets + normalisation pipeline
jefffffff9 +1: Claude Sonnet 4.6- src/data/adlam.py: full Adlam↔Latin transliterator for Pular (Guinea Fula) — bidirectional char map (U+1E900–U+1E95F), digraph-aware latin_to_adlam(), contains_adlam(), normalize_pular() (Adlam→Latin + NFC + lowercase) - src/data/web_harvester.py: add 3 Fula training sources to HF_DATASET_REGISTRY: Pullo-Africa-Protagonist/Fula-pular (9,761 audio rows, primary), guizme/adlam_fulfulde (51 Adlam-script rows, auto-converted to Latin) - notebooks/kaggle_master_trainer.ipynb: Cell 11 gets inline _normalize_pular (Adlam→Latin); Cell 12 applies it for lang=='ful' in prepare_dataset - app.py: import normalize_pular; apply in _do_asr and _convo_pipeline for Fula; update Self-Teaching tab help text to list all 3 Fula datasets Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>ced078c
-
Fix: add trust_remote_code=True for datasets 3.x compatibility
Broulaye Doumbia71bb3bc
-
Phase 3: Voice-to-Voice S2S pipeline — F5-TTS, LLM brain, CER metric
jefffffff9 +1: Claude Sonnet 4.6app.py: - Add LLM_MODEL_ID env var (default: Qwen/Qwen2.5-72B-Instruct) - Import GemmaClient + bam_normalize at module level - Add _voice_ref_path / _voice_ref_text / _llm_client state - Add set_voice_reference(): converts MP3->24kHz WAV, auto-transcribes - Add _convo_pipeline(): ASR -> bam_normalize -> LLM (phonetic Bambara system prompt) -> F5-TTS (voice ref) with MMS-TTS fallback - handle_ask() accepts convo_mode=bool, routes to _convo_pipeline or the original sensor pipeline accordingly - Tab 1 UI: Conversation Mode toggle, Voice Reference upload accordion, stop_recording auto-submit for true back-to-back loop src/tts/f5_tts.py (new): - Lazy-loaded F5TTS wrapper; synthesize() with ref_wav + ref_text - to_wav_24k() resampler (F5-TTS needs 24 kHz input) - Graceful fallback (returns None) when f5-tts not installed src/data/bam_normalize.py (new): - _bam_norm(): ou->u, dj->j, gn->ny, ch->c, oo->o-open, ee->e-open - Used at inference (app.py) and training (notebook Cell 11) req…8952fff
-
Fix Self-Teaching tab: float sliders, deduplication, Kaggle API fallback
jefffffff9 +1: Claude Sonnet 4.6- Cast slider values to int (Gradio returns float; broke Wikipedia aplimit and Kaggle ds.take() in notebook Cell 9) - Deduplicate dataset_sources.jsonl on each HF import (avoid loading same dataset multiple times per training run) - Kaggle trigger: try Python API (KaggleApiExtended) first to avoid binary PATH issues; subprocess + shutil.which as fallback - Fix Fula dataset registry: fleurs uses legacy script loader incompatible with datasets>=3.0; switch to google/WaxalNLP ful_asr - Fix UI description: HF import writes to dataset_sources.jsonl (streamed by Kaggle), not to corrections.jsonl Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>24b1617 -
Fix Wikipedia and HF harvest buttons
jefffffff9 +1: Claude Sonnet 4.6Wikipedia fix: - Add User-Agent header to all requests (Wikipedia API returns 403 without it) - Extract shared _upload_jsonl() helper to reduce code duplication HF harvest fix: - Remove audio re-upload approach (500 HTTP requests -> Gradio timeout) - Instead write dataset references to dataset_sources.jsonl on the Hub - One small file upload (~instant) instead of gigabytes of audio - Kaggle notebook Cell 4 now reads dataset_sources.jsonl dynamically: any dataset registered in Space Self-Teaching tab is automatically picked up at training time with no notebook edits needed Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>a5737ac -
Add mass training + internet self-teaching features
jefffffff9 +1: Claude Sonnet 4.6New Tab 5 — Bulk Upload: - Upload ZIP of audio files + CSV (filename,transcription) in one operation - Batch-inserts all samples into corrections.jsonl + uploads audio to Hub New Tab 6 — Self-Teaching: - Wikipedia harvest: bm/ff Wikipedia API → sentence text → vocabulary.jsonl (868 Bambara articles, 17k+ Fula articles) - HF dataset import: RobotsMali/jeli-asr (33k Bambara) + google/fleurs ff_sn (Fula) → corrections.jsonl, one click - Auto-training trigger: fires Kaggle kernel via REST API when corrections.jsonl reaches AUTO_TRAIN_THRESHOLD (default 50) entries Requires KAGGLE_USERNAME + KAGGLE_KEY Space secrets New src/data/web_harvester.py: - harvest_wikipedia_text(lang, max_articles) — MediaWiki API, no deps - harvest_hf_audio(lang, token) — generator yielding (wav_bytes, text, path) - _numpy_to_wav_bytes() — stdlib-only WAV encoder Auto-trigger also fires after each Tab 2 correction save. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>eddcaea
-
Add multi-user speaker profiles, collective voice, and mode toggle
jefffffff9 +1: Claude Sonnet 4.6Task 1 — SpeakerProfileManager (src/voice/speaker_profiles.py): SpeechBrain ECAPA-TDNN identifies speakers by cosine similarity of 192-d embeddings (threshold 0.75). Each user gets user_N_sb.npy (running-average SpeechBrain embedding) and user_N_ov.npy (running-average OpenVoice V2 tone-color SE). New utterances update the running average so profiles sharpen over time. Task 2 — get_collective_embedding() (speaker_profiles.py): Loads all user_N_ov.npy files and returns their mean vector — the "Collective Voice" that blends all known speakers into one SE. Task 3 — Voice Mode toggle (app_lab.py + src/tts/voice_cloner.py): Added Individual / Collective radio to the UI. In Individual mode the TTS output is cloned into the last detected speaker's voice via OpenVoice V2 ToneColorConverter. In Collective mode the mean SE is used. Text-input path falls back to the collective SE. Both modes degrade gracefully to base VITS voice if OpenVoice is not ready. requirem…49910a9 -
Add confidence loop, curiosity engine, and lightweight TTS
jefffffff9 +1: Claude Sonnet 4.6Task 1 — TTS refactor (src/tts/waxal_tts.py): Switch Bambara TTS from Qwen2-based MALIBA-AI to ynnov/ekodi-bambara-tts-female (VitsModel + AutoTokenizer) — much lighter on CPU Basic; no trust_remote_code needed. Fula is an explicit generate_pular_tts() placeholder returning None until the model is trained. Task 2 — Active learning / confidence loop (src/engine/stt_processor.py): transcribe_with_confidence() wraps Whisper generate() with output_scores=True and computes avg_logprob via compute_transition_scores(). If avg_logprob < -1.0, app_lab.py replaces the transcript with CONFUSION_PROMPT so the LLM asks the user in English to repeat and explain the word. Task 3 — Proactive gaps (src/engine/curiosity.py): CuriosityEngine.maybe_ask() fires every 5 interactions — sends the last 10 vocabulary entries to Qwen and appends a 🌱 question in the chat asking the user to teach a missing agricultural term. Task 4 — Zero-cost persistence: MemoryManager._push_to_h…3657607 -
Fix Bambara TTS red state + surface detailed errors in UI
jefffffff9 +1: Claude Sonnet 4.6- src/tts/waxal_tts.py: load MALIBA-AI/bambara-tts directly via AutoModelForCausalLM (trust_remote_code=True) — no pip install needed at runtime; HF Spaces blocks GitHub outbound so the old lazy subprocess install was silently failing every time - app_lab.py: wrap process_audio / process_text in try/except so exceptions surface as '❌ Error: ...' in the status box instead of a generic Gradio popup with no message; add logging Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>082adaa -
Fix: install maliba-ai lazily at runtime, not at build time
jefffffff9 +1: Claude Sonnet 4.6maliba-ai 2.0.0b0 has strict pins (soundfile==0.13.1, librosa==0.11.0) that conflict with our requirements.txt. Removing it from requirements.txt and installing it via subprocess on the first Bambara TTS call instead. This pattern is standard for HF Spaces when a dependency has irreconcilable version conflicts with the rest of the stack. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>c154b17 -
Phase 2: Waxal TTS — Bambara voice output + Fula training notebook
jefffffff9 +1: Claude Sonnet 4.6TTS engine (src/tts/waxal_tts.py): - Bambara: MALIBA-AI/bambara-tts (non-Meta, Mali community, 10 native speakers) Loads via custom maliba-ai package; writes WAV to tempfile, reads back as numpy - Fula: ous-sow/fula-tts (our own model, loads once trained) Lazy-loads; gracefully reports 'not trained yet' until notebook is run - WaxalTTSEngine.audio_to_gradio() converts float32 → int16 for gr.Audio app_lab.py: - Imports WaxalTTSEngine; preloads both models at startup in background - _run_llm_and_tts() shared core: Gemma → memory → TTS → return audio tuple - process_audio() and process_text() now return 4-tuple (adds audio_out) - UI: added gr.Audio output widget with autoplay; status bar shows TTS readiness per language (🟢/🟡/🔴) Training notebook (notebooks/train_fula_tts.ipynb): - 9 cells: GPU check → install → HF login → config → load WaxalNLP ful_tts → prepare dataset (WAV + metadata.csv) → Coqui VITS trainer → push to HF Hub → synthesis test - Runs on Kaggle T4 (~2-3h); …dd38e25 -
Fix: switch LLM to Qwen2.5-72B-Instruct (Gemma not on HF free tier)
jefffffff9 +1: Claude Sonnet 4.6google/gemma-3-4b-it requires a paid provider on HF Serverless Inference. Qwen/Qwen2.5-72B-Instruct is available on the free tier and has better multilingual coverage for Bambara/Fula context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>61e52d7 -
Phase 1: Sahel-Voice-Lab — The Memory Loop
jefffffff9 +1: Claude Sonnet 4.6New project pivot: self-learning voice assistant for Bambara/Fula using 100% non-Meta tech stack. New files: - app_lab.py — Gradio UI: push-to-talk + last 5 words panel - src/memory/memory_manager.py — persists vocabulary.jsonl to HF Hub - src/llm/gemma_client.py — Gemma via HF Serverless Inference API - data/vocabulary.jsonl — empty initial vocabulary store Flow: audio → Whisper STT → Gemma (with vocabulary context) → teaching intent → MemoryManager.add_word_pair() → Hub push question intent → answer from vocabulary conversation → natural reply README.md updated: app_file changed to app_lab.py, stack documented. Stack: openai/whisper-large-v3-turbo (STT), google/gemma-3-4b-it (LLM), Waxal TTS (Phase 2), HF Dataset for memory. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>096b19d
-
Add phrase library + fuzzy matcher for general Bambara/Fula conversation
jefffffff9 +1: Claude Sonnet 4.6Core problem: the app responded to 'I ni ce' with a sensor default message because it had no conversational knowledge. Fixed with a phrase-library approach that requires zero ML training. New files: - data/phrases/bambara.json — 37 curated Bambara phrases with Whisper variants - data/phrases/fula.json — 38 curated Fula/Pular phrases with Whisper variants - src/conversation/phrase_matcher.py — fuzzy matcher using rapidfuzz (falls back to substring match if rapidfuzz unavailable) app.py changes: - Pipeline tries phrase library FIRST; sensor pipeline is fallback - Low-confidence unknown intents return 'I didn't understand' in native lang - New 'Knowledge Base' tab: paste phrase pairs (native|english) — immediate - YouTube import tab: download audio + save transcription to training dataset - User additions persisted to HF Hub (survive Space restarts) - Phrase additions loaded from Hub in background at startup requirements.txt: added rapidfuzz==3.13.0, yt-dlp==2025.3.31 Co-Author…6f4d8d0 -
Improve Bambara/Fula quality + add full correction UI
jefffffff9 +1: Claude Sonnet 4.6voice_responder.py: - Every template now returns (native_text, english_translation) tuple - Added greeting/farewell/thanks/soil_ok/weather_ok/pest_ok templates - Short MMS-TTS-optimised sentences kept (≤6 words) intent_parser.py: - Added greeting, thanks, farewell intents with Bambara/Fula/French/English keywords - Added French and English keywords to agricultural intents app.py: - _run_pipeline returns (transcript, english_translation, response_text, audio) - Tab 1: added English translation box; added "Send to Correction" button - Tab 2: added English translation field + corrected_english field + corrected_response field - "Send to Correction" auto-populates all Tab 2 fields from Tab 1 in one click - _save_feedback_to_hub stores corrected_english and corrected_response Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>d351193 -
Switch training dataset from google/waxal to google/fleurs
jefffffff9 +1: Claude Sonnet 4.6google/waxal was removed from HuggingFace Hub. google/fleurs is the maintained replacement with identical column schema (audio, transcription). Subset mapping: bam → bam_ML, ful → ff_SN. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>d2183cd
-
Initial commit: Sahel-Agri Voice AI
jefffffff976db545