Spaces:
Sleeping
Sleeping
Commit History
-
Phase 3: Voice-to-Voice S2S pipeline — F5-TTS, LLM brain, CER metric
jefffffff9 +1: Claude Sonnet 4.6app.py: - Add LLM_MODEL_ID env var (default: Qwen/Qwen2.5-72B-Instruct) - Import GemmaClient + bam_normalize at module level - Add _voice_ref_path / _voice_ref_text / _llm_client state - Add set_voice_reference(): converts MP3->24kHz WAV, auto-transcribes - Add _convo_pipeline(): ASR -> bam_normalize -> LLM (phonetic Bambara system prompt) -> F5-TTS (voice ref) with MMS-TTS fallback - handle_ask() accepts convo_mode=bool, routes to _convo_pipeline or the original sensor pipeline accordingly - Tab 1 UI: Conversation Mode toggle, Voice Reference upload accordion, stop_recording auto-submit for true back-to-back loop src/tts/f5_tts.py (new): - Lazy-loaded F5TTS wrapper; synthesize() with ref_wav + ref_text - to_wav_24k() resampler (F5-TTS needs 24 kHz input) - Graceful fallback (returns None) when f5-tts not installed src/data/bam_normalize.py (new): - _bam_norm(): ou->u, dj->j, gn->ny, ch->c, oo->o-open, ee->e-open - Used at inference (app.py) and training (notebook Cell 11) req…8952fff
-
Add multi-user speaker profiles, collective voice, and mode toggle
jefffffff9 +1: Claude Sonnet 4.6Task 1 — SpeakerProfileManager (src/voice/speaker_profiles.py): SpeechBrain ECAPA-TDNN identifies speakers by cosine similarity of 192-d embeddings (threshold 0.75). Each user gets user_N_sb.npy (running-average SpeechBrain embedding) and user_N_ov.npy (running-average OpenVoice V2 tone-color SE). New utterances update the running average so profiles sharpen over time. Task 2 — get_collective_embedding() (speaker_profiles.py): Loads all user_N_ov.npy files and returns their mean vector — the "Collective Voice" that blends all known speakers into one SE. Task 3 — Voice Mode toggle (app_lab.py + src/tts/voice_cloner.py): Added Individual / Collective radio to the UI. In Individual mode the TTS output is cloned into the last detected speaker's voice via OpenVoice V2 ToneColorConverter. In Collective mode the mean SE is used. Text-input path falls back to the collective SE. Both modes degrade gracefully to base VITS voice if OpenVoice is not ready. requirem…49910a9 -
Add confidence loop, curiosity engine, and lightweight TTS
jefffffff9 +1: Claude Sonnet 4.6Task 1 — TTS refactor (src/tts/waxal_tts.py): Switch Bambara TTS from Qwen2-based MALIBA-AI to ynnov/ekodi-bambara-tts-female (VitsModel + AutoTokenizer) — much lighter on CPU Basic; no trust_remote_code needed. Fula is an explicit generate_pular_tts() placeholder returning None until the model is trained. Task 2 — Active learning / confidence loop (src/engine/stt_processor.py): transcribe_with_confidence() wraps Whisper generate() with output_scores=True and computes avg_logprob via compute_transition_scores(). If avg_logprob < -1.0, app_lab.py replaces the transcript with CONFUSION_PROMPT so the LLM asks the user in English to repeat and explain the word. Task 3 — Proactive gaps (src/engine/curiosity.py): CuriosityEngine.maybe_ask() fires every 5 interactions — sends the last 10 vocabulary entries to Qwen and appends a 🌱 question in the chat asking the user to teach a missing agricultural term. Task 4 — Zero-cost persistence: MemoryManager._push_to_h…3657607 -
Fix Bambara TTS red state + surface detailed errors in UI
jefffffff9 +1: Claude Sonnet 4.6- src/tts/waxal_tts.py: load MALIBA-AI/bambara-tts directly via AutoModelForCausalLM (trust_remote_code=True) — no pip install needed at runtime; HF Spaces blocks GitHub outbound so the old lazy subprocess install was silently failing every time - app_lab.py: wrap process_audio / process_text in try/except so exceptions surface as '❌ Error: ...' in the status box instead of a generic Gradio popup with no message; add logging Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>082adaa -
Fix: install maliba-ai lazily at runtime, not at build time
jefffffff9 +1: Claude Sonnet 4.6maliba-ai 2.0.0b0 has strict pins (soundfile==0.13.1, librosa==0.11.0) that conflict with our requirements.txt. Removing it from requirements.txt and installing it via subprocess on the first Bambara TTS call instead. This pattern is standard for HF Spaces when a dependency has irreconcilable version conflicts with the rest of the stack. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>c154b17 -
Phase 2: Waxal TTS — Bambara voice output + Fula training notebook
jefffffff9 +1: Claude Sonnet 4.6TTS engine (src/tts/waxal_tts.py): - Bambara: MALIBA-AI/bambara-tts (non-Meta, Mali community, 10 native speakers) Loads via custom maliba-ai package; writes WAV to tempfile, reads back as numpy - Fula: ous-sow/fula-tts (our own model, loads once trained) Lazy-loads; gracefully reports 'not trained yet' until notebook is run - WaxalTTSEngine.audio_to_gradio() converts float32 → int16 for gr.Audio app_lab.py: - Imports WaxalTTSEngine; preloads both models at startup in background - _run_llm_and_tts() shared core: Gemma → memory → TTS → return audio tuple - process_audio() and process_text() now return 4-tuple (adds audio_out) - UI: added gr.Audio output widget with autoplay; status bar shows TTS readiness per language (🟢/🟡/🔴) Training notebook (notebooks/train_fula_tts.ipynb): - 9 cells: GPU check → install → HF login → config → load WaxalNLP ful_tts → prepare dataset (WAV + metadata.csv) → Coqui VITS trainer → push to HF Hub → synthesis test - Runs on Kaggle T4 (~2-3h); …dd38e25
-
Initial commit: Sahel-Agri Voice AI
jefffffff976db545