File size: 3,634 Bytes
191645f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | # Project Context
## Purpose
Build an "Accelerated Book Copilot" Streamlit app that ingests long-form content, delivers layered summaries, enables adjustable-speed listening, answers reader questions with cited passages, and enriches context via targeted web crawling.
## Tech Stack
- Python 3.11+
- Streamlit for the interactive UI
- LangChain for orchestration, retrieval, and tool management
- Azure OpenAI / OpenAI models for summarization and Q&A
- SentenceTransformers or `text-embedding-3-small` embeddings
- FAISS (local) or Pinecone (managed) vector store
- TTS utilities from FOMO project (gTTS / local models) with optional Azure Speech
- Newspaper3k / Requests / BeautifulSoup for focused crawling
- Altair for visualization (timeline, reading metrics)
## Project Conventions
### Code Style
- Pythonic style with type hints, descriptive snake_case naming, and minimal inline comments.
- Prefer pure functions where practical; separate UI logic from data/LLM orchestration.
- Use `black`/`ruff` compatible formatting (4-space indent, double quotes acceptable but prefer single quotes unless escaping).
- Avoid interactive `input()`; rely on Streamlit widgets and session state.
### Architecture Patterns
- Layered architecture: ingestion → processing (chunk/embedding/summarization) → experience (UI tabs).
- Retrieval-Augmented Generation for Q&A with citation packaging.
- Background task pattern for heavy preprocessing (async threads or queued jobs).
- Modular service helpers (ingestion, summarization, retrieval, TTS) to keep Streamlit pages thin.
### Testing Strategy
- Unit tests for ingestion utilities, chunking, and summarization prompt builders.
- Integration smoke tests for the Streamlit app using sample documents.
- Mock external APIs (Azure OpenAI, Pinecone) to keep CI deterministic.
- Manual regression checklist for multilingual TTS and crawl workflows before demos.
### Git Workflow
- Use feature branches named `feature/<topic>` or `fix/<topic>` off `main`.
- Conventional commit messages (e.g., `feat: add crawl summarizer`) with co-author trailer when pair hacking.
- Pull requests require reviewer sign-off plus local smoke test evidence.
- Avoid committing secrets; rely on `.env` templates and local dotenv loading.
## Domain Context
- Focused on long-form content acceleration for readers, researchers, and students.
- Supports multiple summary granularities (flash, detailed, character/plot focused).
- Q&A must cite source passages to maintain trust; responses should match question language when possible.
- Listening mode ties TTS playback to highlighted text segments for faster comprehension.
- Crawl mode gathers external commentary (reviews, interviews, scholarly notes) to enrich understanding.
## Important Constraints
- Must operate within hackathon time constraints; prioritize features that demo well within 3 days.
- Respect licensing of ingested books; app assumes user-provided content is permitted.
- Ensure OpenAI/Azure usage stays within provided quota; implement caching to minimize token spend.
- Crawling restricted to whitelisted, publicly accessible sources to avoid legal issues.
- Offline-friendly fallback path (local embeddings/FAISS) required if Pinecone credentials unavailable.
## External Dependencies
- Azure OpenAI / OpenAI API keys for LLM and embedding calls.
- Pinecone account (optional) for managed vector storage.
- Hugging Face / SentenceTransformers models for local embeddings.
- Third-party TTS backends (gTTS/local models/Azure Speech) for audio output.
- Newspaper3k/Requests for external article retrieval; BeautifulSoup for parsing.
|