README.md

metadata

title: FamilyLegacy
emoji: 📊
colorFrom: gray
colorTo: indigo
sdk: gradio
sdk_version: 6.17.3
python_version: '3.12'
app_file: app.py
pinned: true
license: mit
short_description: AI-driven voice cloning to preserve the voices of loved ones
tags:
  - track:backyard
  - sponsor:openbmb
  - achievement:offgrid
  - achievement:offbrand
  - achievement:sharing
  - achievement:fieldnotes

🔗 Resources & Links

• Demo Video: Watch the Product Demo
• Blog Post: Read the Full Write-up
• Linkedin Post Read the post

https://huggingface.co/pkheria

🌟 Project Overview

FamilyLegacy is a compassionate AI application designed to bridge the gap between generations. By leveraging state-of-the-art voice cloning and large language models, it allows users to preserve the voices and stories of their loved ones, creating a digital legacy that can "talk back" with warmth and personality.

✨ Key Features

• Voice Cloning: Capture a short sample of a loved one's voice to create a high-fidelity digital twin using VoxCPM2.
• Memory Storage: Upload audio recordings of family stories. The system transcribes them and stores them in a vector database (Qdrant) for semantic retrieval.
• Heartwarming Interactions: Ask questions or prompt the AI to tell a story. The system retrieves relevant memories and generates a personalized response in the cloned voice.
• Emotionally Aware: Responses are tailored to the relationship (e.g., grandmother, father, friend) with specific tonal traits like "warm," "wise," or "cheerful."
• Privacy Focused: Designed to be hosted securely, ensuring your family's precious data and voices remain under your control.

🛠️ Tech Stack

• Backend: FastAPI & Gradio
• Voice Synthesis: VoxCPM2
• Text Generation: Qwen2.5-3B-Instruct
• Transcription: OpenAI Whisper
• Vector Database: Qdrant (for story memory retrieval)
• Primary Database: MongoDB (for voice profiles and metadata)
• Embeddings: sentence-transformers/all-MiniLM-L6-v2

🏗️ Architecture

The application follows a modern AI pipeline:

1. Ingestion: Voice samples are saved to MongoDB. Story audio is transcribed via Whisper and indexed into Qdrant.
2. Retrieval: When a user asks a question, the system searches Qdrant for relevant family memories.
3. Generation: An LLM (Qwen) takes the retrieved context and generates a heartwarming response.
4. Synthesis: The generated text is passed to VoxCPM2 along with the reference voice profile to produce the final audio output.

🚀 Getting Started

Prerequisites

• Python 3.10+
• MongoDB instance
• Qdrant instance
• FFmpeg (for audio processing)

Installation

1. Clone the repository:

   git clone <repository-url>
   cd FAMILY-LEGACY
   

2. Set up a virtual environment:

   python -m venv .venv
   source .venv/bin/activate  # On Windows use `.venv\Scripts\activate`
   

3. Install dependencies:

   pip install -r requirements.txt
   

4. Environment Configuration: Create a .env file in the root directory and add your connection strings (do not share these!):

   MONGO_URL=your_mongodb_connection_string
   MONGO_DB_NAME=family_legacy
   QDRANT_URL=your_qdrant_url
   QDRANT_API_KEY=your_qdrant_api_key
   

Running the App

python app.py

The application will be available at http://localhost:7860.

📜 License

This project is licensed under the MIT License - see the header of this file for details.