README.md

---
title: FamilyLegacy
emoji: 📊
colorFrom: gray
colorTo: indigo
sdk: gradio
sdk_version: 6.17.3
python_version: '3.12'
app_file: app.py
pinned: true
license: mit
short_description: AI-driven voice cloning to preserve the voices of loved ones
tags:
  - track:backyard
  - sponsor:openbmb
  - achievement:offgrid
  - achievement:offbrand
  - achievement:sharing
  - achievement:fieldnotes
---

## 🔗 Resources & Links

- **Demo Video:** [Watch the Product Demo]([YOUR_DEMO_VIDEO_LINK_HERE](https://youtu.be/e02qP1gLHVI))
- **Blog Post:** [Read the Full Write-up](https://huggingface.co/blog/pkheria/familylegacy)
- **Linkedin Post** [Read the post](https://www.linkedin.com/posts/piyushkheria7_buildsmall-opensource-generativeai-share-7472298281763770368-En6K/)

https://huggingface.co/pkheria

## 🌟 Project Overview

**FamilyLegacy** is a compassionate AI application designed to bridge the gap between generations. By leveraging state-of-the-art voice cloning and large language models, it allows users to preserve the voices and stories of their loved ones, creating a digital legacy that can "talk back" with warmth and personality.

## ✨ Key Features

- **Voice Cloning:** Capture a short sample of a loved one's voice to create a high-fidelity digital twin using `VoxCPM2`.
- **Memory Storage:** Upload audio recordings of family stories. The system transcribes them and stores them in a vector database (`Qdrant`) for semantic retrieval.
- **Heartwarming Interactions:** Ask questions or prompt the AI to tell a story. The system retrieves relevant memories and generates a personalized response in the cloned voice.
- **Emotionally Aware:** Responses are tailored to the relationship (e.g., grandmother, father, friend) with specific tonal traits like "warm," "wise," or "cheerful."
- **Privacy Focused:** Designed to be hosted securely, ensuring your family's precious data and voices remain under your control.

## 🛠️ Tech Stack

- **Backend:** [FastAPI](https://fastapi.tiangolo.com/) & [Gradio](https://gradio.app/)
- **Voice Synthesis:** [VoxCPM2](https://github.com/OpenBMB/VoxCPM)
- **Text Generation:** [Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
- **Transcription:** [OpenAI Whisper](https://github.com/openai/whisper)
- **Vector Database:** [Qdrant](https://qdrant.tech/) (for story memory retrieval)
- **Primary Database:** [MongoDB](https://www.mongodb.com/) (for voice profiles and metadata)
- **Embeddings:** `sentence-transformers/all-MiniLM-L6-v2`

## 🏗️ Architecture

The application follows a modern AI pipeline:
1.  **Ingestion:** Voice samples are saved to MongoDB. Story audio is transcribed via Whisper and indexed into Qdrant.
2.  **Retrieval:** When a user asks a question, the system searches Qdrant for relevant family memories.
3.  **Generation:** An LLM (Qwen) takes the retrieved context and generates a heartwarming response.
4.  **Synthesis:** The generated text is passed to VoxCPM2 along with the reference voice profile to produce the final audio output.

## 🚀 Getting Started

### Prerequisites

- Python 3.10+
- MongoDB instance
- Qdrant instance
- FFmpeg (for audio processing)

### Installation

1.  **Clone the repository:**
    ```bash
    git clone <repository-url>
    cd FAMILY-LEGACY
    ```

2.  **Set up a virtual environment:**
    ```bash
    python -m venv .venv
    source .venv/bin/activate  # On Windows use `.venv\Scripts\activate`
    ```

3.  **Install dependencies:**
    ```bash
    pip install -r requirements.txt
    ```

4.  **Environment Configuration:**
    Create a `.env` file in the root directory and add your connection strings (do not share these!):
    ```env
    MONGO_URL=your_mongodb_connection_string
    MONGO_DB_NAME=family_legacy
    QDRANT_URL=your_qdrant_url
    QDRANT_API_KEY=your_qdrant_api_key
    ```

### Running the App

```bash
python app.py
```
The application will be available at `http://localhost:7860`.

## 📜 License

This project is licensed under the MIT License - see the header of this file for details.