Spaces:
Paused
Paused
Sumedh
Claude Sonnet 4.6
Claude: add MeloTTS Indian voice, onboarding personalization, class-based speed
4821868 | # IDEAS.md — HopeV1.0 Future Ideas | |
| Ideas to explore and implement when time permits. | |
| Add new ideas here during conversations so nothing gets lost. | |
| --- | |
| ## TTS & Voice | |
| - [ ] **Try ElevenLabs voice** — "Sarah" or "Aria" voice sounds like a real teacher. | |
| Free tier: 10K chars/month. Paid: $5/month for 30K chars. Best quality available. | |
| Integration: replace `kokoro_tts_to_wav()` with ElevenLabs API call. | |
| - [ ] **Try OpenAI TTS "nova" voice** — Very natural, warm. Pay-per-use ($0.015/1K chars). | |
| Good middle ground between Kokoro (free) and ElevenLabs (best). | |
| - [ ] **Emotion-aware TTS** — Detect if the LLM reply is praise vs. question vs. correction, | |
| and adjust TTS speed/tone accordingly. Kokoro supports speed tuning per-segment. | |
| - [ ] **Child voice profiles** — Let teacher set age group (5–7, 8–10, 10–12) which adjusts | |
| story complexity, question difficulty, and TTS speed automatically. | |
| --- | |
| ## Stories & Content | |
| - [ ] **Story library expansion** — Currently 7 hardcoded stories. Load stories from a | |
| JSON/CSV file so teachers can add new ones without touching code. | |
| - [ ] **Grade-level tagging** — Tag each story with difficulty level so the bot can pick | |
| an appropriate story based on student's grade. | |
| - [ ] **Curriculum alignment** — Align stories and questions to specific English curriculum | |
| standards (e.g., CBSE, ICSE). | |
| - [ ] **Multi-language support** — Whisper already supports Hindi→English translation. | |
| Could add regional language greetings/encouragement while keeping English content. | |
| --- | |
| ## Session & Pedagogy | |
| - [ ] **Grammar correction mode (2 phases)** | |
| Phase 1 (current): Bot is encouraging only. Does NOT correct grammar. | |
| Phase 2 (future): After N sessions or when confidence is high, bot identifies grammatical | |
| errors in the student's speech and gently asks them to repeat correctly. | |
| **Important implementation note:** Whisper (ASR) transcribes speech as-is and does NOT | |
| auto-correct grammar — it faithfully passes through mistakes (e.g. "I goed to school" | |
| stays as-is). This is correct behaviour. The Qwen LLM is what should identify errors. | |
| The `fixed_prompt` in `coach_core.py` controls this — currently it says nothing about | |
| grammar correction. | |
| To implement Phase 2: add a `correction_mode: bool` flag to session state, and modify | |
| the per-session user message to instruct Qwen to gently point out one grammar mistake | |
| per turn and ask for a repeat. | |
| - [ ] **Adaptive questioning** — If child answers incorrectly, ask a simpler follow-up | |
| instead of moving to the next question. Requires tracking answer quality in state. | |
| - [ ] **Progress tracking** — Save session history per phone number (story read, | |
| questions asked, accuracy, date). Expose a teacher dashboard. | |
| - [ ] **Pronunciation feedback** — Use Whisper confidence scores to detect when a child | |
| is struggling to pronounce words and gently correct. | |
| - [ ] **Session summary to teacher** — At end of session, send a WhatsApp text summary | |
| to a teacher's number: "Student answered 2/3 questions correctly. Story: Hare & Turtle." | |
| --- | |
| ## Infrastructure | |
| - [ ] **Move bridge to Railway.app** *(HIGH PRIORITY — planned)* | |
| Current HuggingFace bridge has intermittent SSL issues with Meta's API and | |
| goes to sleep on free tier. Railway.app fixes both: | |
| - No outbound SSL issues | |
| - Always-on (no sleep) | |
| - Cost: ~$0.50/month (well within $5 free credit) | |
| Steps: | |
| 1. Push `hope-whatsapp-bridge` to GitHub repo | |
| 2. Connect to Railway → Deploy from GitHub | |
| 3. Add env vars: VERIFY_TOKEN, WHATSAPP_TOKEN, PHONE_NUMBER_ID, GPU_SERVER_URL, HF_TOKEN | |
| 4. Get Railway domain → update Meta webhook URL | |
| 5. Revert `send_audio()` to direct upload method (no link workaround needed) | |
| 6. Remove `/audio/{filename}` serve endpoint (no longer needed) | |
| - [ ] **Persistent session storage** — Current sessions saved to `/tmp/` which is lost | |
| on Space restart. Use HuggingFace dataset repo or a small SQLite DB on a | |
| persistent volume for session state. | |
| - [ ] **Gradio Space upgrade** — Currently on Gradio 4.36.1; latest is 4.44+. Upgrade | |
| carefully — check that `api_name="whatsapp"` still works after upgrade. | |
| - [ ] **WhatsApp message queueing** — If GPU server is busy, queue incoming messages | |
| and process in order. Currently parallel requests can conflict. | |
| --- | |
| ## Meta / WhatsApp | |
| - [ ] **Permanent WhatsApp token** — Move from temporary 24h token to permanent | |
| System User token via Meta Business Settings. (See CLAUDE.md for steps.) | |
| - [ ] **Meta Business Verification** — Required to message any user (not just test numbers). | |
| Allows real students to start sessions by messaging the number. | |
| Submit at: business.facebook.com → Business Verification. | |
| - [ ] **WhatsApp template messages** — For business-initiated conversations (bot starts | |
| the session, not the student), need approved message templates from Meta. | |
| --- | |
| ## UI / Admin | |
| - [ ] **Teacher web dashboard** — Simple web UI to: view active sessions, see transcripts, | |
| add new stories, adjust number of questions per story. | |
| - [ ] **QR code onboarding** — Generate a WhatsApp QR code that students can scan to | |
| start their first session automatically (sends "start" on their behalf). | |