Spaces:
Running
Running
| title: NotebookLM | |
| emoji: 🚀 | |
| colorFrom: purple | |
| colorTo: blue | |
| sdk: gradio | |
| sdk_version: "5.12.0" | |
| python_version: "3.12" | |
| app_file: app.py | |
| hf_oauth: true | |
| hf_oauth_expiration_minutes: 480 | |
| tags: | |
| - gradio | |
| pinned: false | |
| short_description: NotebookLM - AI-Powered Study Companion | |
| license: mit | |
| # NotebookLM — AI-Powered Study Companion | |
| A full-featured, Google NotebookLM-inspired study tool built with **Gradio** on **Hugging Face Spaces**. Upload documents, chat with your sources using RAG, and generate study artifacts like summaries, podcasts, and quizzes — all from one interface. | |
| --- | |
| ## Features | |
| ### Chat with Your Sources (RAG) | |
| - Ask questions about uploaded documents and get grounded, cited answers | |
| - Two-stage retrieval: semantic vector search (top-40) followed by hybrid reranking (top-8) combining embedding similarity with lexical keyword overlap | |
| - Inline citation chips (`[S1]`, `[S2]`) link back to source passages | |
| - Conversational context: last 5 messages included for follow-up questions | |
| - Powered by **Qwen/Qwen2.5-7B-Instruct** via HF Inference API | |
| ### Multi-Format Document Ingestion | |
| | Format | Extractor | Notes | | |
| |--------|-----------|-------| | |
| | **PDF** | PyMuPDF (fitz) | Full-page text extraction | | |
| | **PPTX** | python-pptx | Text from all slides and shapes | | |
| | **TXT** | Built-in | UTF-8 plain text | | |
| | **Web URLs** | BeautifulSoup | Strips nav/footer/scripts, extracts article content | | |
| | **YouTube** | youtube-transcript-api | Auto-fetches video transcripts | | |
| - Max file size: **15 MB** | |
| - Max sources per notebook: **20** | |
| - Duplicate detection for both files and URLs | |
| ### Ingestion Pipeline | |
| ``` | |
| Upload/URL → Text Extraction → Recursive Chunking → Embedding Generation → Pinecone Upsert | |
| ``` | |
| - **Chunking:** Recursive character splitting (2000 chars per chunk, 200 char overlap) with separators `\n\n` → `\n` → `. ` → ` ` | |
| - **Embeddings:** `sentence-transformers/all-MiniLM-L6-v2` (384 dimensions) via HF Inference API | |
| - **Vector Store:** Pinecone with namespace-per-notebook isolation | |
| ### Document Summary | |
| - Summarize content from your uploaded sources using semantic retrieval | |
| - **Source selection:** Choose specific sources to include via checkbox selector | |
| - **Styles:** Brief (3-5 bullets, <150 words) or Detailed (sectioned, 300-500 words) | |
| - Powered by **Claude Haiku 4.5** | |
| ### Conversation Summary | |
| - Summarize your chat history into structured notes | |
| - **Styles:** Brief or Detailed | |
| - Powered by **Claude Haiku 4.5** | |
| ### Podcast Generation | |
| - Generates a natural two-host dialogue script (Alex & Sam) from your summary | |
| - Script converted to audio via **OpenAI TTS** (`tts-1` model, `alloy` voice) | |
| - In-browser audio player with download option | |
| - Falls back to script-only if TTS is unavailable | |
| ### Quiz Generation | |
| - Creates multiple-choice quizzes (5 or 10 questions) from your source material | |
| - Interactive HTML with "Show Answer" reveal buttons | |
| - Includes explanations for each correct answer | |
| - Downloadable as standalone HTML | |
| ### Multi-Notebook Support | |
| - Create, rename, and delete notebooks from the sidebar | |
| - Each notebook has its own sources, chat history, and artifacts | |
| - Switch between notebooks instantly | |
| ### Authentication & Persistence | |
| - **OAuth** via Hugging Face (session expiration: 480 minutes) | |
| - User data serialized to JSON and stored in a **private HF Dataset repo** (`Group-1-5010/notebooklm-data`) | |
| - Manual save button with unsaved-changes warning on page unload | |
| --- | |
| ## Architecture | |
| ``` | |
| NotebookLM/ | |
| ├── app.py # Gradio UI, event wiring, refresh logic | |
| ├── state.py # Data models: UserData, Notebook, Source, Message, Artifact | |
| ├── theme.py # Dark theme, custom CSS, SVG logos | |
| ├── mock_data.py # Mock responses for offline testing | |
| ├── requirements.txt # Python dependencies | |
| │ | |
| ├── artifacts/ | |
| │ └── prompt.poml # RAG system prompt (POML format) | |
| │ | |
| ├── assets/ | |
| │ ├── logo.svg # App logo (SVG with gradients) | |
| │ └── podcasts/ # Generated podcast MP3 files | |
| │ | |
| ├── ingestion_engine/ # Document processing pipeline | |
| │ ├── ingestion_manager.py # Orchestrates: extract → chunk → embed → upsert | |
| │ ├── pdf_extractor.py # PDF text extraction (PyMuPDF) | |
| │ ├── text_extractor.py # Plain text file reading | |
| │ ├── url_scrapper.py # Web page scraping (BeautifulSoup) | |
| │ ├── transcripter.py # YouTube transcript fetching | |
| │ ├── chunker.py # Recursive text chunking with overlap | |
| │ └── embedding_generator.py # Sentence-transformer embeddings (HF API) | |
| │ | |
| ├── persistence/ # Storage layer | |
| │ ├── vector_store.py # Pinecone CRUD (upsert, query, delete) | |
| │ └── storage_service.py # HF Dataset repo for user data persistence | |
| │ | |
| ├── services/ # Core AI features | |
| │ ├── rag_engine.py # RAG pipeline: retrieve → rerank → generate | |
| │ ├── summary_service.py # Conversation & document summaries (Claude) | |
| │ ├── podcast_service.py # Podcast script generation + OpenAI TTS | |
| │ └── quiz_service.py # Quiz generation + interactive HTML renderer | |
| │ | |
| └── pages/ # UI tab handlers | |
| ├── chat.py # Chat interface with citation rendering | |
| ├── sources.py # Source upload, URL add, delete, status display | |
| └── artifacts.py # Summary, podcast, quiz display & generation | |
| ``` | |
| --- | |
| ## Tech Stack | |
| | Layer | Technology | | |
| |-------|-----------| | |
| | **UI Framework** | Gradio 5.12.0 | | |
| | **Hosting** | Hugging Face Spaces | | |
| | **Auth** | HF OAuth | | |
| | **Chat LLM** | Qwen/Qwen2.5-7B-Instruct (HF Inference API) | | |
| | **Summary / Podcast / Quiz LLM** | Claude Haiku 4.5 (Anthropic API) | | |
| | **Text-to-Speech** | OpenAI TTS (`tts-1`, `alloy` voice) | | |
| | **Embeddings** | sentence-transformers/all-MiniLM-L6-v2 (HF Inference API) | | |
| | **Vector Database** | Pinecone | | |
| | **User Data Storage** | HF Dataset repo (private JSON) | | |
| | **PDF Extraction** | PyMuPDF | | |
| | **PPTX Extraction** | python-pptx | | |
| | **Web Scraping** | BeautifulSoup4 + Requests | | |
| | **YouTube Transcripts** | youtube-transcript-api | | |
| --- | |
| ## Setup | |
| ### Prerequisites | |
| - Python 3.12+ | |
| - A Hugging Face account | |
| - API keys for Pinecone, Anthropic, and OpenAI | |
| ### Environment Variables | |
| Set these as **Secrets** in your HF Space settings (or in a `.env` for local dev): | |
| | Variable | Description | | |
| |----------|-------------| | |
| | `HF_TOKEN` | Hugging Face API token (read/write access) | | |
| | `Pinecone_API` | Pinecone API key | | |
| | `ANTHROPIC_API_KEY` | Anthropic API key (for Claude Haiku) | | |
| | `OPENAI_API_KEY` | OpenAI API key (for TTS audio generation) | | |
| ### Local Development | |
| ```bash | |
| # Clone the repo | |
| git clone https://huggingface.co/spaces/Group-1-5010/NotebookLM | |
| cd NotebookLM | |
| # Install dependencies | |
| pip install -r requirements.txt | |
| # Export your API keys | |
| export HF_TOKEN="hf_..." | |
| export Pinecone_API="..." | |
| export ANTHROPIC_API_KEY="sk-ant-..." | |
| export OPENAI_API_KEY="sk-..." | |
| # Run the app | |
| python app.py | |
| ``` | |
| The app launches at `http://localhost:7860`. | |
| ### Deploying to HF Spaces | |
| 1. Create a new Gradio Space on Hugging Face | |
| 2. Push the code to the Space repo | |
| 3. Add all four API keys as Secrets in the Space settings | |
| 4. The Space auto-builds and deploys | |
| --- | |
| ## How It Works | |
| ### RAG Pipeline (Chat) | |
| ``` | |
| User Question | |
| │ | |
| ├─→ Embed query (MiniLM-L6-v2, 384d) | |
| │ | |
| ├─→ Pinecone semantic search (top-40 candidates) | |
| │ | |
| ├─→ Hybrid rerank: embedding_score + 0.05 × keyword_overlap | |
| │ └─→ Deduplicate → top-8 final chunks | |
| │ | |
| ├─→ Build prompt (system rules + context + last 5 messages + question) | |
| │ | |
| └─→ Qwen 2.5-7B-Instruct generates grounded answer with [S1], [S2] citations | |
| ``` | |
| ### Artifact Generation | |
| ``` | |
| Sources (Pinecone) ──→ Claude Haiku ──→ Summary (Markdown) | |
| │ | |
| ├──→ Claude Haiku ──→ Podcast Script | |
| │ │ | |
| │ OpenAI TTS ──→ MP3 Audio | |
| │ | |
| └──→ Claude Haiku ──→ Quiz (JSON → Interactive HTML) | |
| ``` | |
| ### Data Flow | |
| ``` | |
| Upload File ──→ Extract Text ──→ Chunk (500 tokens, 50 overlap) | |
| │ | |
| ├──→ Embed (HF Inference API) | |
| │ | |
| └──→ Upsert to Pinecone (namespace = notebook_id) | |
| metadata: {source_id, source_filename, chunk_index, text} | |
| ``` | |
| --- | |
| ## UI Overview | |
| ### Sidebar | |
| - Create / rename / delete notebooks | |
| - Notebook selector (radio buttons with source & message counts) | |
| - Save button with unsaved-changes indicator | |
| ### Chat Tab | |
| - Chatbot with message bubbles and citation chips | |
| - Warning banner if no sources are uploaded yet | |
| ### Sources Tab | |
| - Drag-and-drop file uploader (PDF, PPTX, TXT) | |
| - URL input for web pages and YouTube videos | |
| - Source cards showing type icon, file size, chunk count, and status badge | |
| - Status indicators: Processing (yellow pulse), Ready (green), Failed (red with tooltip) | |
| - Delete source dropdown | |
| ### Artifacts Tab | |
| **Summary Sub-tab:** | |
| - Conversation Summary: style selector (brief/detailed) + generate button | |
| - Document Summary: source selector (checkboxes) + style selector + generate button | |
| - Download buttons for each (`.md`) | |
| **Podcast Sub-tab:** | |
| - Locked until a summary is generated | |
| - Generate button produces dialogue script + MP3 audio | |
| - In-browser audio player | |
| - Download button (`.mp3`) | |
| **Quiz Sub-tab:** | |
| - Question count selector (5 or 10) | |
| - Interactive multiple-choice with "Show Answer" reveals | |
| - Download button (`.html`) | |
| --- | |
| ## Design | |
| - **Theme:** Custom dark theme (Indigo/Purple gradient) | |
| - **Background:** `#0e1117` | |
| - **Font:** Inter (Google Fonts) | |
| - **Color palette:** Indigo (`#667eea`), Purple (`#764ba2`), Gold accent (`#fbbf24`), Green success (`#22c55e`) | |
| - **Custom SVG logo** with notebook + sparkle motif | |
| --- | |
| ## Dependencies | |
| ``` | |
| gradio>=5.0.0 | |
| huggingface_hub>=0.20.0 | |
| pinecone>=5.0.0 | |
| PyMuPDF>=1.23.0 | |
| python-pptx>=0.6.21 | |
| beautifulsoup4>=4.12.0 | |
| requests>=2.31.0 | |
| youtube-transcript-api>=0.6.0 | |
| scipy>=1.11.0 | |
| anthropic>=0.40.0 | |
| openai>=1.0.0 | |
| ``` | |
| --- | |
| ## License | |
| MIT | |