Hugging Face's logo

Spaces:
HopeLLM
/
HopeV1.0
Paused

App Files Community
2
HopeV1.0 / IDEAS.md
Sumedh
Claude: add MeloTTS Indian voice, onboarding personalization, class-based speed
4821868
preview code
|
Raw History Blame Contribute Delete
5.26 kB

IDEAS.md — HopeV1.0 Future Ideas

Ideas to explore and implement when time permits. Add new ideas here during conversations so nothing gets lost.

TTS & Voice

  • Try ElevenLabs voice — "Sarah" or "Aria" voice sounds like a real teacher. Free tier: 10K chars/month. Paid: $5/month for 30K chars. Best quality available. Integration: replace kokoro_tts_to_wav() with ElevenLabs API call.

  • Try OpenAI TTS "nova" voice — Very natural, warm. Pay-per-use ($0.015/1K chars). Good middle ground between Kokoro (free) and ElevenLabs (best).

  • Emotion-aware TTS — Detect if the LLM reply is praise vs. question vs. correction, and adjust TTS speed/tone accordingly. Kokoro supports speed tuning per-segment.

  • Child voice profiles — Let teacher set age group (5–7, 8–10, 10–12) which adjusts story complexity, question difficulty, and TTS speed automatically.

Stories & Content

  • Story library expansion — Currently 7 hardcoded stories. Load stories from a JSON/CSV file so teachers can add new ones without touching code.

  • Grade-level tagging — Tag each story with difficulty level so the bot can pick an appropriate story based on student's grade.

  • Curriculum alignment — Align stories and questions to specific English curriculum standards (e.g., CBSE, ICSE).

  • Multi-language support — Whisper already supports Hindi→English translation. Could add regional language greetings/encouragement while keeping English content.

Session & Pedagogy

  • Grammar correction mode (2 phases) Phase 1 (current): Bot is encouraging only. Does NOT correct grammar. Phase 2 (future): After N sessions or when confidence is high, bot identifies grammatical errors in the student's speech and gently asks them to repeat correctly.

    Important implementation note: Whisper (ASR) transcribes speech as-is and does NOT auto-correct grammar — it faithfully passes through mistakes (e.g. "I goed to school" stays as-is). This is correct behaviour. The Qwen LLM is what should identify errors. The fixed_prompt in coach_core.py controls this — currently it says nothing about grammar correction.

    To implement Phase 2: add a correction_mode: bool flag to session state, and modify the per-session user message to instruct Qwen to gently point out one grammar mistake per turn and ask for a repeat.

  • Adaptive questioning — If child answers incorrectly, ask a simpler follow-up instead of moving to the next question. Requires tracking answer quality in state.

  • Progress tracking — Save session history per phone number (story read, questions asked, accuracy, date). Expose a teacher dashboard.

  • Pronunciation feedback — Use Whisper confidence scores to detect when a child is struggling to pronounce words and gently correct.

  • Session summary to teacher — At end of session, send a WhatsApp text summary to a teacher's number: "Student answered 2/3 questions correctly. Story: Hare & Turtle."

Infrastructure

  • Move bridge to Railway.app (HIGH PRIORITY — planned) Current HuggingFace bridge has intermittent SSL issues with Meta's API and goes to sleep on free tier. Railway.app fixes both:

    • No outbound SSL issues
    • Always-on (no sleep)
    • Cost: ~$0.50/month (well within $5 free credit) Steps:
    1. Push hope-whatsapp-bridge to GitHub repo
    2. Connect to Railway → Deploy from GitHub
    3. Add env vars: VERIFY_TOKEN, WHATSAPP_TOKEN, PHONE_NUMBER_ID, GPU_SERVER_URL, HF_TOKEN
    4. Get Railway domain → update Meta webhook URL
    5. Revert send_audio() to direct upload method (no link workaround needed)
    6. Remove /audio/{filename} serve endpoint (no longer needed)

  • Persistent session storage — Current sessions saved to /tmp/ which is lost on Space restart. Use HuggingFace dataset repo or a small SQLite DB on a persistent volume for session state.

  • Gradio Space upgrade — Currently on Gradio 4.36.1; latest is 4.44+. Upgrade carefully — check that api_name="whatsapp" still works after upgrade.

  • WhatsApp message queueing — If GPU server is busy, queue incoming messages and process in order. Currently parallel requests can conflict.

Meta / WhatsApp

  • Permanent WhatsApp token — Move from temporary 24h token to permanent System User token via Meta Business Settings. (See CLAUDE.md for steps.)

  • Meta Business Verification — Required to message any user (not just test numbers). Allows real students to start sessions by messaging the number. Submit at: business.facebook.com → Business Verification.

  • WhatsApp template messages — For business-initiated conversations (bot starts the session, not the student), need approved message templates from Meta.

UI / Admin

  • Teacher web dashboard — Simple web UI to: view active sessions, see transcripts, add new stories, adjust number of questions per story.

  • QR code onboarding — Generate a WhatsApp QR code that students can scan to start their first session automatically (sends "start" on their behalf).